https://homeostasis.scs.carleton.ca/wiki/api.php?action=feedcontributions&feedformat=atom&user=SpankeSoma-notes - User contributions [en]2026-05-01T17:08:55ZUser contributionsMediaWiki 1.42.1https://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_2_2010_Question_11&diff=6196Talk:COMP 3000 Essay 2 2010 Question 112010-12-02T05:21:02Z<p>Spanke: </p>
<hr />
<div>--[[User:Spanke|Spanke]] 00:19, 2 December 2010 (UTC) Finished Timers, I hate 3004...<br />
<br />
--[[User:Jjpwilso|Jjpwilso]] 00:03, 2 December 2010 (UTC) I'll check for some more references on the ACM and IEEE databases. In the meantime I thought I'd mention what Anil said regarding critique. He suggested we should consider other approaches to the same solution, such as modifying NTP with a different heuristic. I'll see what I can dig up in other papers on NTP.<br />
<br />
--[[User:ScottG|ScottG]] 22:06, 1 December 2010 (UTC) I'm assuming you meant for me to add my references, yes? I really only used the article, and 'Timekeeping in Virtual Machines' which I went to add, but is already on there. I've looked for other articles to try to get how others have looked at it that aren't VMware, but there really isn't a huge amount out there dealing ''specifically'' with guest timekeeping (unless I've gone Google-blind, which has admittedly happened before). Mostly I ran into links pointing to that specific article.<br />
<br />
--[[User:Sblais2|Sblais2]] 17:50, 1 December 2010 (UTC) I added stuf into the Research problems. I think I summarized most of them. If I forgot any, please add them in. I also added the missing references in the reference section. For Fedor, we seem to miss some content in 2 sections. Also, you could read through the other section and add/change some pertinent information that might've been missed that would make this essay even better.<br />
<br />
--[[User:Sblais2|Sblais2]] 15:11, 1 December 2010 (UTC) Would it be possible to add your references at the bottom please? Even if it is a link. I have added the article link at the top of the essay.<br />
<br />
Hey guys, sorry its taken me a while to post here. If there is a particular topic that needs researching, I could spend some hours doing that tomorrow - suggestions? Also, I intend to fix up the style&structure after everything is done as I am quite good with that. <br />
<br />
Fedor<br />
<br />
--[[User:ScottG|ScottG]] 21:36, 26 November 2010 (UTC) So I was a little (more than a little) behind on my initially estimated time for getting stuff up on Guest Timekeeping, but that's the gist of it there now. I'm going to try to buff it up a bit before it's due, since what I put in is a bit rougher than I'd like. If I seem to be missing something that should be pretty obvious, let me know and I'll work it in.<br />
<br />
--[[User:Jjpwilso|Jjpwilso]] 15:49, 23 November 2010 (UTC) I've been completely swamped with COMP3004 stuff (among other things) and feeling guilty as hell about this essay. The good news, for those who might have missed today's lecture, is we have an extension of one week. Phew!!<br />
<br />
--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.<br />
<br />
--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em: <br />
<br>http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br>http://www.xen.org/files/xen_interface.pdf<br />
<br>http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx<br />
<br>http://www.intel.com/hardwaredesign/hpetspec_1.pdf<br />
<br>http://www.cubinlab.ee.unimelb.edu.au/radclock/<br />
<br />
--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.<br />
<br />
--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.<br />
<br />
--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.<br />
<br />
<br />
=Participants=<br />
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com<br><br />
(X) Graham Scott sgraham6<br><br />
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com <br> <br />
(X) Panke Shane spanke<br><br />
(X) Shukla Abhinav ashukla2<br><br />
(X) Wilson Robert jjpwilso<br></div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=6190COMP 3000 Essay 2 2010 Question 112010-12-02T04:59:39Z<p>Spanke: /* Timers */</p>
<hr />
<div>= Virtualize Everything But Time =<br />
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]<br />
<br />
<br />
=Background Concepts=<br />
<br />
The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.<br />
<br />
The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]<br />
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]<br />
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
<br />
Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]<br />
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work. <br />
<br />
Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]<br />
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.<br />
<br />
In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.<br />
<br />
=Contribution=<br />
<br />
(sections are stubs for the moment ... more to come)<br />
The contributions of this paper were:<br />
<br />
* baseline evaluations of:<br />
** performance of NTP in dependent and independent configurations<br />
** Xen Clocksource as a basis counter under NTP<br />
** latencies of different clock sources<br />
** implications of Power Management<br />
<br />
* new architecture<br />
** RADclock<br />
** XenStore as holder of clock parameter data<br />
** feed-forward versus feedback<br />
<br />
* evaluation of RADclock vs ntpd<br />
<br />
=Critique=<br />
=References=<br />
<br />
1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf<br />
<br />
2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm <br />
<br />
4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html<br />
<br />
5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf<br />
<br />
6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf<br />
<br />
7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf<br />
<br />
8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=6169COMP 3000 Essay 2 2010 Question 112010-12-02T04:05:55Z<p>Spanke: /* Timers */</p>
<hr />
<div>= Virtualize Everything But Time =<br />
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]<br />
<br />
<br />
=Background Concepts=<br />
<br />
The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.<br />
<br />
The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]<br />
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]<br />
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
<br />
Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]<br />
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work. <br />
<br />
Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]<br />
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.<br />
<br />
In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.<br />
<br />
=Contribution=<br />
<br />
(sections are stubs for the moment ... more to come)<br />
The contributions of this paper were:<br />
<br />
* baseline evaluations of:<br />
** performance of NTP in dependent and independent configurations<br />
** Xen Clocksource as a basis counter under NTP<br />
** latencies of different clock sources<br />
** implications of Power Management<br />
<br />
* new architecture<br />
** RADclock<br />
** XenStore as holder of clock parameter data<br />
** feed-forward versus feedback<br />
<br />
* evaluation of RADclock vs ntpd<br />
<br />
=Critique=<br />
=References=<br />
<br />
1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf<br />
<br />
2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm <br />
<br />
4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html<br />
<br />
5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf<br />
<br />
6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf<br />
<br />
7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=6145COMP 3000 Essay 2 2010 Question 112010-12-02T03:36:46Z<p>Spanke: /* Timekeeping */</p>
<hr />
<div>= Virtualize Everything But Time =<br />
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]<br />
<br />
<br />
=Background Concepts=<br />
<br />
The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.<br />
<br />
The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]<br />
# TSC<br />
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
<br />
Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]<br />
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work. <br />
<br />
Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]<br />
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.<br />
<br />
In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.<br />
<br />
=Contribution=<br />
<br />
(sections are stubs for the moment ... more to come)<br />
The contributions of this paper were:<br />
<br />
* baseline evaluations of:<br />
** performance of NTP in dependent and independent configurations<br />
** Xen Clocksource as a basis counter under NTP<br />
** latencies of different clock sources<br />
** implications of Power Management<br />
<br />
* new architecture<br />
** RADclock<br />
** feed-forward versus feedback<br />
<br />
* evaluation of RADclock vs ntpd<br />
<br />
=Critique=<br />
=References=<br />
<br />
1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf<br />
<br />
2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm <br />
<br />
4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html<br />
<br />
5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf<br />
<br />
6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf<br />
<br />
7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=6139COMP 3000 Essay 2 2010 Question 112010-12-02T03:34:05Z<p>Spanke: /* Timers */</p>
<hr />
<div>= Virtualize Everything But Time =<br />
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]<br />
<br />
<br />
=Background Concepts=<br />
<br />
The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.<br />
<br />
The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]<br />
# TSC<br />
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
<br />
Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]<br />
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work. <br />
<br />
Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]<br />
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.<br />
<br />
In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.<br />
<br />
=Contribution=<br />
<br />
(sections are stubs for the moment ... more to come)<br />
The contributions of this paper were:<br />
<br />
* baseline evaluations of:<br />
** performance of NTP in dependent and independent configurations<br />
** Xen Clocksource as a basis counter under NTP<br />
** latencies of different clock sources<br />
** implications of Power Management<br />
<br />
* new architecture<br />
** RADclock<br />
** feed-forward versus feedback<br />
<br />
* evaluation of RADclock vs ntpd<br />
<br />
=Critique=<br />
=References=<br />
<br />
1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf<br />
<br />
2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm <br />
<br />
4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html<br />
<br />
5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf<br />
<br />
6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf<br />
<br />
7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=6129COMP 3000 Essay 2 2010 Question 112010-12-02T03:25:20Z<p>Spanke: /* Timers */</p>
<hr />
<div>= Virtualize Everything But Time =<br />
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]<br />
<br />
<br />
=Background Concepts=<br />
<br />
The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.<br />
<br />
The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/]<br />
# TSC<br />
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
<br />
Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]<br />
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work. <br />
<br />
Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]<br />
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.<br />
<br />
In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.<br />
<br />
=Contribution=<br />
<br />
(sections are stubs for the moment ... more to come)<br />
The contributions of this paper were:<br />
<br />
* baseline evaluations of:<br />
** performance of NTP in dependent and independent configurations<br />
** Xen Clocksource as a basis counter under NTP<br />
** latencies of different clock sources<br />
** implications of Power Management<br />
<br />
* new architecture<br />
** RADclock<br />
** feed-forward versus feedback<br />
<br />
* evaluation of RADclock vs ntpd<br />
<br />
=Critique=<br />
=References=<br />
<br />
1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf<br />
<br />
2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm <br />
<br />
4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html<br />
<br />
5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf<br />
<br />
6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf<br />
<br />
7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5693COMP 3000 Essay 2 2010 Question 112010-11-29T17:41:20Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]<br />
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5691COMP 3000 Essay 2 2010 Question 112010-11-29T16:57:31Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]<br />
# Local APIC (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5690COMP 3000 Essay 2 2010 Question 112010-11-29T16:47:56Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC, also known as a CMOS battery, allows the system to keep time even while the physical PC unit is unplugged, or the laptop is out of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM.<br />
# Local APIC (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
<br />
Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.<br />
<br />
===Sources of Drift===<br />
<br />
When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.<br />
<br />
Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.<br />
<br />
===Impact of Drift===<br />
<br />
The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.<br />
<br />
===Compensation Strategies===<br />
<br />
There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.<br />
<br />
If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5430COMP 3000 Essay 2 2010 Question 112010-11-23T04:28:03Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC (I will fill these in later) --[[User:Spanke|Spanke]]<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5429COMP 3000 Essay 2 2010 Question 112010-11-23T04:27:29Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC (I will fill these in later) - Shane<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5428COMP 3000 Essay 2 2010 Question 112010-11-23T04:27:01Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC (I will fill these in later)<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5427COMP 3000 Essay 2 2010 Question 112010-11-23T04:26:41Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. We only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC (I will fill these in later)<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5426COMP 3000 Essay 2 2010 Question 112010-11-23T04:22:47Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC (I will fill these in later)<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_2_2010_Question_11&diff=5425COMP 3000 Essay 2 2010 Question 112010-11-23T04:22:21Z<p>Spanke: /* Timers */</p>
<hr />
<div>=Background Concepts=<br />
<br />
To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.<br />
<br />
===Timekeeping===<br />
<br />
Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.<br />
<br />
===Clocks===<br />
<br />
Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:<br />
<br />
Diagram1. Timer Abstraction<br />
<br />
[[File:Timerabstract.jpg]]<br />
<br />
This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.<br />
<br />
===Timers===<br />
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. (I will fill these in later) [http://www.osdever.net/bkerndev/Docs/pit.htm]<br />
# CMOS RTC<br />
# Local APIC<br />
# ACPI<br />
# TSC<br />
# HPET<br />
<br />
==Guest Timekeeping==<br />
This section will explain why timing is different for guest operating systems.<br />
===Sources of Drift===<br />
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.<br />
===Impact of Drift===<br />
Discuss the impact of error on a few real-world applications.<br />
===Compensation Strategies===<br />
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.<br />
<br />
=Research problem=<br />
=Contribution=<br />
=Critique=<br />
=References=<br />
<br />
1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf<br />
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_2_2010_Question_11&diff=4965Talk:COMP 3000 Essay 2 2010 Question 112010-11-15T03:05:34Z<p>Spanke: </p>
<hr />
<div>Please mark an X if you are able to participate.<br />
<br />
(X) Blais Sylvain sblais2<br><br />
(X) Graham Scott sgraham6<br><br />
( ) Ilitchev Fedor filitche<br><br />
(X) Panke Shane spanke<br><br />
(X) Shukla Abhinav ashukla2<br><br />
(X) Wilson Robert jjpwilso<br></div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=4288COMP 3000 Essay 1 2010 Question 72010-10-15T00:39:21Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
== The Background ==<br />
<br />
A 'process' is defined to be "an address-space and a group of resources dedicated to running the program". On the other hand a 'thread' is an independent sequential unit of computation that executes within the context of a kernel supported entity like a 'process'. Threads are often classified by their “weight” (or overhead), which corresponds to the amount of context that must be saved when a thread is removed from the processor, and restored when a thread is reinstated on a processor that is a context switch. The context for a process usually includes the hardware register, kernel stack, user-level stack, interrupt vectors, page tables, and more. Threads require less system resources then concurrent cooperating processes and start much easier, therefore, there may exist millions of them in a single process. Loosely based on this there are two major types of threads: kernel and user-mode. Kernel threads are usually considered heavier and designs that involve them are not very scalable. User threads, on the other hand, are mapped to kernel threads and lightwieght. The ratio of the user threads to kernel threads is an important factor when designing scalable systems.<br />
<br />
There are a few designs, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools, which are yet another mechanism that allows for high scalability. Systems can support millions of threads within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br><br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the basic challenges is to create code which is stable and at the same time scalable. Furthermore, the challenge in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. Ray Bryant and John Hawkes found the following bottlenecks when porting Linux to a 64-core NUMA system. Each of these bottle necks are an example of a type of bottleneck that can appear in any program.[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux, there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive when compared to what would happen if the information was in the 'right place'. Once the misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times, this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' An example of one such bottleneck is the xtime_lock in Linux. Having locking read prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. This was deemed an unacceptable state of affairs. So, the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems: the spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. The Scheduler also had O(n) time complexity which essentially meant that the scheduler had scalability issues and would become inefficient after a particular number of processes. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler with a O(1)time complexity which meant that any number of threads/processes could be scheduled without any overhead. <br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements that handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines, however, they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed, they sleep or wait. This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can, for instance, handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices == <br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' <br />
This is the most basic design and the lightweight process. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Management and scheduling is done through thread management. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model. Thread aware operating system is found on Windows XP, Windows 2000, Windows Vista and any latest operating system.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation.<br />
<br />
Locks are an important part of the performance of threads and there are multiple way of controlling and creating locks in order to create a large amount of threads. Single lock (having the data structures all be in one lock) has the advantage that once the processor has acquired the lock it can modify any of the stored data. Using the single lock method means only one lock is needed per thread, decreasing the thread overhead but this also limits the throughput of the system. Multiple lock (having each data structure have it's own lock) has the advantage of that each action on the data structure is it's own locking/unlocking operations. Multiple has greater thread overhead (because there are more locks) but the thread throughput is much higher allowing for fast creation of threads. Another downside of multiple lock systems are deadlocks, a deadlock happens when two different threads are waiting for data that the other task holds. Single and multiple lock systems are the inverse of each other and using both depending on the situation can greatly increase the performance of a system. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/], Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
Linux Symposium, pg83 [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br><br />
PicoThreads: Lightweight Threads in Java [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=4287COMP 3000 Essay 1 2010 Question 72010-10-15T00:39:06Z<p>Spanke: /* Scalable Threads: The Problems */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
== The Background ==<br />
<br />
A 'process' is defined to be "an address-space and a group of resources dedicated to running the program". On the other hand a 'thread' is an independent sequential unit of computation that executes within the context of a kernel supported entity like a 'process'. Threads are often classified by their “weight” (or overhead), which corresponds to the amount of context that must be saved when a thread is removed from the processor, and restored when a thread is reinstated on a processor that is a context switch. The context for a process usually includes the hardware register, kernel stack, user-level stack, interrupt vectors, page tables, and more. Threads require less system resources then concurrent cooperating processes and start much easier, therefore, there may exist millions of them in a single process. Loosely based on this there are two major types of threads: kernel and user-mode. Kernel threads are usually considered heavier and designs that involve them are not very scalable. User threads, on the other hand, are mapped to kernel threads and lightwieght. The ratio of the user threads to kernel threads is an important factor when designing scalable systems.<br />
<br />
There are a few designs, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools, which are yet another mechanism that allows for high scalability. Systems can support millions of threads within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br><br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the basic challenges is to create code which is stable and at the same time scalable. Furthermore, the challenge in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. Ray Bryant and John Hawkes found the following bottlenecks when porting Linux to a 64-core NUMA system. Each of these bottle necks are an example of a type of bottleneck that can appear in any program.[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux, there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive when compared to what would happen if the information was in the 'right place'. Once the misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times, this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' An example of one such bottleneck is the xtime_lock in Linux. Having locking read prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. This was deemed an unacceptable state of affairs. So, the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems: the spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. The Scheduler also had O(n) time complexity which essentially meant that the scheduler had scalability issues and would become inefficient after a particular number of processes. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler with a O(1)time complexity which meant that any number of threads/processes could be scheduled without any overhead. <br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements that handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines, however, they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed, they sleep or wait. This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can, for instance, handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices == <br />
--[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' <br />
This is the most basic design and the lightweight process. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Management and scheduling is done through thread management. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model. Thread aware operating system is found on Windows XP, Windows 2000, Windows Vista and any latest operating system.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation.<br />
<br />
Locks are an important part of the performance of threads and there are multiple way of controlling and creating locks in order to create a large amount of threads. Single lock (having the data structures all be in one lock) has the advantage that once the processor has acquired the lock it can modify any of the stored data. Using the single lock method means only one lock is needed per thread, decreasing the thread overhead but this also limits the throughput of the system. Multiple lock (having each data structure have it's own lock) has the advantage of that each action on the data structure is it's own locking/unlocking operations. Multiple has greater thread overhead (because there are more locks) but the thread throughput is much higher allowing for fast creation of threads. Another downside of multiple lock systems are deadlocks, a deadlock happens when two different threads are waiting for data that the other task holds. Single and multiple lock systems are the inverse of each other and using both depending on the situation can greatly increase the performance of a system. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/], Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
Linux Symposium, pg83 [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br><br />
PicoThreads: Lightweight Threads in Java [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=4286COMP 3000 Essay 1 2010 Question 72010-10-15T00:38:27Z<p>Spanke: /* The Background */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
== The Background ==<br />
<br />
A 'process' is defined to be "an address-space and a group of resources dedicated to running the program". On the other hand a 'thread' is an independent sequential unit of computation that executes within the context of a kernel supported entity like a 'process'. Threads are often classified by their “weight” (or overhead), which corresponds to the amount of context that must be saved when a thread is removed from the processor, and restored when a thread is reinstated on a processor that is a context switch. The context for a process usually includes the hardware register, kernel stack, user-level stack, interrupt vectors, page tables, and more. Threads require less system resources then concurrent cooperating processes and start much easier, therefore, there may exist millions of them in a single process. Loosely based on this there are two major types of threads: kernel and user-mode. Kernel threads are usually considered heavier and designs that involve them are not very scalable. User threads, on the other hand, are mapped to kernel threads and lightwieght. The ratio of the user threads to kernel threads is an important factor when designing scalable systems.<br />
<br />
There are a few designs, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools, which are yet another mechanism that allows for high scalability. Systems can support millions of threads within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br><br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the basic challenges is to create code which is stable and at the same time scalable. Furthermore, the challenge in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. Ray Bryant and John Hawkes found the following bottlenecks when porting Linux to a 64-core NUMA system. Each of these bottle necks are an example of a type of bottleneck that can appear in any program.[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux, there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive when compared to what would happen if the information was in the 'right place'. Once the misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times, this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' An example of one such bottleneck is the xtime_lock in Linux. Having locking read prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. This was deemed an unacceptable state of affairs. So, the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems: the spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. The Scheduler also had O(n) time complexity which essentially meant that the scheduler had scalability issues and would become inefficient after a particular number of processes. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler with a O(1)time complexity which meant that any number of threads/processes could be scheduled without any overhead. <br />
--[[Rannath]] A few additions--[[Gautam]] --Cache-coherency is not the important part --[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements that handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines, however, they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed, they sleep or wait. This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can, for instance, handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices == <br />
--[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' <br />
This is the most basic design and the lightweight process. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Management and scheduling is done through thread management. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model. Thread aware operating system is found on Windows XP, Windows 2000, Windows Vista and any latest operating system.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation.<br />
<br />
Locks are an important part of the performance of threads and there are multiple way of controlling and creating locks in order to create a large amount of threads. Single lock (having the data structures all be in one lock) has the advantage that once the processor has acquired the lock it can modify any of the stored data. Using the single lock method means only one lock is needed per thread, decreasing the thread overhead but this also limits the throughput of the system. Multiple lock (having each data structure have it's own lock) has the advantage of that each action on the data structure is it's own locking/unlocking operations. Multiple has greater thread overhead (because there are more locks) but the thread throughput is much higher allowing for fast creation of threads. Another downside of multiple lock systems are deadlocks, a deadlock happens when two different threads are waiting for data that the other task holds. Single and multiple lock systems are the inverse of each other and using both depending on the situation can greatly increase the performance of a system. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/], Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
Linux Symposium, pg83 [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83]<br><br />
PicoThreads: Lightweight Threads in Java [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3935COMP 3000 Essay 1 2010 Question 72010-10-14T18:44:07Z<p>Spanke: /* Answer */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]] && [[Shane]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the basic challenges is to create code which is stable and at the same time scalable. Furthermore, the challenge in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. Ray Bryant and John Hawkes found the following bottlenecks when porting Linux to a 64-core NUMA system. Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "'''cache-coherency operation'''" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices == <br />
--[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' <br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/], Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3847COMP 3000 Essay 1 2010 Question 72010-10-14T16:25:32Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/], Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3846COMP 3000 Essay 1 2010 Question 72010-10-14T16:19:16Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3845COMP 3000 Essay 1 2010 Question 72010-10-14T16:18:50Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
{{refs|2}}</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3844COMP 3000 Essay 1 2010 Question 72010-10-14T16:16:59Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
<references/></div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3843COMP 3000 Essay 1 2010 Question 72010-10-14T16:16:05Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<ref>{{cite web|url=http://uk.pc.ign.com/articles/999/999171p1.html|title=No LAN for Starcraft II|last=Haynes|first=Jeff|publisher=IGN|accessdate=2009-10-27}}</ref><br />
<br />
== References ==<br />
<references/></div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3842COMP 3000 Essay 1 2010 Question 72010-10-14T16:14:02Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
<references/></div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3841COMP 3000 Essay 1 2010 Question 72010-10-14T16:12:11Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3840COMP 3000 Essay 1 2010 Question 72010-10-14T16:11:55Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
<references/><br />
{{Refimprove|date=June 2008}}<br />
<br />
[[Category:Resources| ]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3839COMP 3000 Essay 1 2010 Question 72010-10-14T16:10:40Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.<ref>[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx ''Scheduling Priorities'']</ref>, Microsoft (23 September 2010) --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3838COMP 3000 Essay 1 2010 Question 72010-10-14T16:09:05Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][2] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3837COMP 3000 Essay 1 2010 Question 72010-10-14T16:06:42Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][2] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
# [http://delivery.acm.org/10.1145/80000/75378/p49-anderson.pdf?key1=75378&key2=7712707821&coll=GUIDE&dl=GUIDE&CFID=105794086&CFTOKEN=35781739 The performance implications of thread management alternatives for shared-memory multiprocessors], ACM (May 1989)<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010)<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009)</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3836COMP 3000 Essay 1 2010 Question 72010-10-14T16:06:24Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][#2] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
# [http://delivery.acm.org/10.1145/80000/75378/p49-anderson.pdf?key1=75378&key2=7712707821&coll=GUIDE&dl=GUIDE&CFID=105794086&CFTOKEN=35781739 The performance implications of thread management alternatives for shared-memory multiprocessors], ACM (May 1989)<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010)<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009)</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3835COMP 3000 Essay 1 2010 Question 72010-10-14T16:05:44Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
# [http://delivery.acm.org/10.1145/80000/75378/p49-anderson.pdf?key1=75378&key2=7712707821&coll=GUIDE&dl=GUIDE&CFID=105794086&CFTOKEN=35781739 The performance implications of thread management alternatives for shared-memory multiprocessors], ACM (May 1989)<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010)<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009)</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3834COMP 3000 Essay 1 2010 Question 72010-10-14T16:05:01Z<p>Spanke: /* References */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==<br />
# [http://delivery.acm.org/10.1145/80000/75378/p49-anderson.pdf?key1=75378&key2=7712707821&coll=GUIDE&dl=GUIDE&CFID=105794086&CFTOKEN=35781739], ACM (May 1989)<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010)<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009)</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3829COMP 3000 Essay 1 2010 Question 72010-10-14T16:00:06Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency.[http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx][http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/] --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3821COMP 3000 Essay 1 2010 Question 72010-10-14T15:51:14Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
<br />
Process is known as an instance of a program running in a computer which has its own resources such as address space, files, I/O devices and thread on the other hand is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously and it can either execute the same code or a different code within the same application because it has its own state, run-time stack and execution context. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability. Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br> [[vG]] && [[Paul]]<br />
<br />
'''Taken the liberty to add Praubic's tentative first para. ''' and '''i have added my version to pauls and modified it [[vG]]'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks once scaled. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them). Each of these bottle necks is an example of a type of bottleneck that can appear in any program.<br />
<br />
When expensive operations are '''needlessly called''' one type of bottleneck appears. In Linux there can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is expensive compared to what would happen if the information was in the 'right place'. Once misplaced information that causes this problem all the time is identified it can be moved to limit the problem. Anywhere expensive operations are called a needless number of times this bottleneck can appear (this problem is not inherent, but is a result of bad-design).<br />
<br />
Another type of bottleneck is from '''starvation.''' One such bottleneck is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, causing the kernel to waste CPU time to keep trying. This problem was solved by using a lockless-read. This problem would appear anywhere that a thread must keep trying to execute, but cannot, leading to wasted CPU cycles.<br />
<br />
The next type of bottleneck is from '''course-grained''' operations. Granularity refers to the execution time of a code segment. Both examples eat alot of CPU time, where a finer-grained implementation would eat less. The closer a segment is to the speed of an atomic action the finer its granularity. One course-grained bottleneck was the dcache_lock. It ate up some time in normal use, but it was also called in the much more popular dnotify_parent() function. That was an unacceptable state of affairs. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux. Another big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottleneck. Both those examples are the result of course granularity. <br />
<br />
Bottlenecks can be from '''multiple problems.''' One example of that is the multiqueue scheduler from linux 2.4. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time, it was course-grained. While, the rest went into computing and recomputing information in the cache, a needless expensive operation. These problems were fixed by replacing the scheduler (That scheduler was then replaced by a more efficient scheduler [O(1) scheduler]).<br />
<br />
--[[Rannath]]<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes. <br />
<br />
Fibers embrace essentially the same abstraction as coroutines. The distinction emerges from the fact that fibers are on the system level while coroutines execute on the language level. Unlike UMS, fibers do not utilize multiprocessor machines however they require less operating system support. Symbian Operating System presents an example of fibers usage in its Active Scheduler. An object of active scheduler contains a single fiber that is scheduled when an asynchronous call returns and blocks lower priority fibers until all above are finished. <br />
<br />
Thread Pools consist of queues of threads that stay open and await new tasks to become assigned to them. If there is no new tasks to be completed they sleep or wait.This pattern eliminates the overhead of creation and destruction of threads which reflects in better system stability and improved performance. The long living threads can for instance handle multiple transaction requests from socket connection from other machines over a short time frame while at the same time avoiding the millions of cycles to drop/reestablish a thread. Often, thread pool operate on server farms and therefore thread-safety has to be carefully implemented.<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
<br />
Explaining the four types of synchronization:<br />
<br />
*Mutex locks uses only a thread thus giving access to only certain part of the code<br />
*Using Read/Write synchronization one can gain exclusive write and read access to protected resource but to edit the content it must have the exclusive write lock. Exclusive write lock is only permitted when all the read locks are released<br />
*Condition variable synchronization protects the thread until the condition becomes true<br />
*Counting semaphores delivers access to multiple threads. It has a count which keeps tracks of the number of threads can have concurrent access to the data. Once the limit is reached other threads are blocked until the limit changes.<br />
[[vG]]<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
<br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
<br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3627Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:42:54Z<p>Spanke: /* Log */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
Answered Essay Questions in Discussion. --[[User:Spanke|Shane]] 01:25, 13 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
# What makes threads non-scalable? List the problems<br />
# What utility do some scalable implementations lack? Why?<br />
# Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# Memory Usage, Context Switching. Consider using a thread pool.<br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
# If using thread pools, the scalability is then limited to the number of threads in the pool<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]] You stated it is possible you did not state how, or rather did not make it clear. The below should be a better interpretation. --[[User:Spanke|Shane]] <br />
<br />
Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3626Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:42:19Z<p>Spanke: /* Log */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
Answered Essay Questions. --[[User:Spanke|Shane]] 01:25, 13 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
# What makes threads non-scalable? List the problems<br />
# What utility do some scalable implementations lack? Why?<br />
# Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# Memory Usage, Context Switching. Consider using a thread pool.<br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
# If using thread pools, the scalability is then limited to the number of threads in the pool<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]] You stated it is possible you did not state how, or rather did not make it clear. The below should be a better interpretation. --[[User:Spanke|Shane]] <br />
<br />
Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3624Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:41:21Z<p>Spanke: /* Essay Outline */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
# What makes threads non-scalable? List the problems<br />
# What utility do some scalable implementations lack? Why?<br />
# Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# Memory Usage, Context Switching. Consider using a thread pool.<br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
# If using thread pools, the scalability is then limited to the number of threads in the pool<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]] You stated it is possible you did not state how, or rather did not make it clear. The below should be a better interpretation. --[[User:Spanke|Shane]] <br />
<br />
Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3620Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:35:22Z<p>Spanke: /* Essay Outline */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
#What makes threads non-scalable? List the problems<br />
#What utility do some scalable implementations lack? Why?<br />
#Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# <br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
#<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]] You stated it is possible you did not state how, or rather did not make it clear. The below should be a better interpretation. --[[User:Spanke|Shane]] <br />
<br />
Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3617Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:34:55Z<p>Spanke: /* Essay Outline */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
#What makes threads non-scalable? List the problems<br />
#What utility do some scalable implementations lack? Why?<br />
#Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# <br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
#<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]] --[[User:Spanke|Shane]] You stated it is possible you did not state how, or rather did not make it clear. The below should be a better interpretation.<br />
<br />
Systems can support millions within a single process by switching execution resources between threads, creating a concurrent execution. Concurrency is the result of multiple threads staying on the queues but is incapable of running them at the same time. It provides the impression that they are executing at the same time due to the speed they switch at.<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3604Talk:COMP 3000 Essay 1 2010 Question 72010-10-14T04:26:11Z<p>Spanke: /* Log */</p>
<hr />
<div>== Log ==<br />
'''Suggestion:''' Let us maintain our edits here instead of on littering the main page with our names. Also please do not edit without writing to the log so that we know who has done what and when.<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity. Everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
More moving for clarity. added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
Edited Scheduling Priorities and rewrote some areas to provide a better paragraph structure. --[[User:Spanke|Shane]] 15:25, 13 October 2010 (UTC)<br />
<br />
Added to the memory management section. --[[User:Hirving|Hirving]] 21:42, 13 October 2010 (UTC)<br />
<br />
Edited Scalable Threads Problems. Also did a little re-arrangement. --[[User:Gautam|Gautam]] 01:03, 14 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
Multi-threading is a term used to describe:<br />
<br />
* A facility provided by the operating system that enables an application to create threads of execution within a process<br />
* Applications whose architecture takes advantage of the multi-threading provided by the operating system <br />
[[vG]]<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
#What makes threads non-scalable? List the problems<br />
#What utility do some scalable implementations lack? Why?<br />
#Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# <br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
#<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --[[User:Praubic|Praubic]] 00:18, 14 October 2010 (UTC)<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
we can add this for intro paragraph:<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process?<br />
<br />
It is possible for systems to supports millions of threads or more within a single processor, it has the ability to switch execution resource between threads, thus making a concurrent execution. Concurrency is when multiple threads stays on the ques for switching but incapable of running at the same time but it has the ability to make it look like they are running at same time due to the speed they switch. [[vG]]<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3600COMP 3000 Essay 1 2010 Question 72010-10-14T04:24:10Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
== Scalable Threads: The Problems ==<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
'''Cache-Coherency:'''There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
'''Locks:'''There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
'''Scheduler:'''The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
'''CPU:'''The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity. Another course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
'''Kernel Lock:'''One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
'''MAIN POINT 2 Paragraph draft''' --[[User:Praubic|Praubic]] 00:21, 14 October 2010 (UTC) still in progress and debating <br />
<br />
Introduction of Windows NT and OS/2 brought about innovation that provides cheap threading while having expensive processing. UMS which reflects such design is a recommended mechanism for high performance requirements which handle many threads on multicore systems. A scheduler has to be implemented to manage the UMS threads and decide when they should be run or stopped. This implementation is not desirable for moderate performance systems because concurrent execution of this sort naturally allows for non-intuitive outcomes or behaviors such as race condition which requires careful programming and design choices. The framework used by UMS threading is divided into smaller abstractions depending on the final desired utility. For instance, UMS scheduling can be assigned to each logical processor and thereby creating affinity for related threads to function around one scheduler. This could turn out inefficient depending whether there are many related threads that could end up starving other processes.<br />
<br />
Ok for point 2 -> I posted a draft on the essay page but Im not certain as to whether i should talk about fibers since they are also functioning on user space but theyre not UMS. --Praubic<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>''' --[[User:Gautam|Gautam]] 00:29, 14 October 2010 (UTC)<br><br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
Thread memory management is an important design choice when attempting to create a large amount of threads in a single process, from creation to maintenance and deallocation. A thread's data structure is made up of a program counter, a stack and a control block. A control block of a thread is needed for thread management as it contains the state data of a thread. The optimization of this data structure can greatly increase performance in large number of threads. <br />
The creation of a thread can take place before the process actually requires it to run and wait until a idle processor becomes available to run the thread. Thread overhead (the required memory, CPU time, and read/write time to initialize the thread) is a problem that can arise with this creation process, since it frontloads the process. Another problem with this creation process is that the thread must allocate the memory required for it's stack at creation because it is expensive to dynamically allocate the stack memory. A way to optimize this creation process for large amounts of threads is to copy the arguments of the thread into it's control block, this allows for the thread's stack to be allocated at the thread's startup (when the thread starts being used) and not when the thread is created. When the thread enters startup it can copy it's arguments out of it's control block and allocate it's memory. Thread creation is ruled by latency (the cost of thread management on the system) and throughput (the rate that the system can create, start, and finish threads that are in contention), and, if thread memory management is done in a serial processing manner, these two factor combine to create a maximum rate of thread creation. <br />
The deallocation of a thread can also be optimized for use in increasing the scalability of threads. Storing deallocted stacks and control blocks in a free list allows the process of allocation and deallocation to be a list operation, if they are not stored in a free list then the thread overhead would include finding the correct size of free memory to store the stack. [http://portal.acm.org/citation.cfm?id=75378] [[hirving]]<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3327COMP 3000 Essay 1 2010 Question 72010-10-13T19:18:29Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC) --[[Spanke|Shane]] '''Revised it a bit, scheduling priority does not use number range in Linux.'''<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3326COMP 3000 Essay 1 2010 Question 72010-10-13T19:17:22Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC) --[[Spanke|Shane]]: '''Revised it a bit, scheduling priority does not use number range in Linux.'''<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3325COMP 3000 Essay 1 2010 Question 72010-10-13T19:15:32Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC) [[Shane]]: '''Revising it a bit Linux does not use the same functionality as Windows.'''<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3324COMP 3000 Essay 1 2010 Question 72010-10-13T19:15:02Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC) [Shane]: Revising it a bit Linux does not use the same functionality as Windows.<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&diff=3322Talk:COMP 3000 Essay 1 2010 Question 72010-10-13T19:12:35Z<p>Spanke: /* Sources */</p>
<hr />
<div>== The Question ==<br />
'''Original:<br>'''<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
<br />
'''Rannath:<br>'''<br />
The question seems to be about number and scalability of threads not the gross mechanics.<br />
<br />
To be more clear: we can limit ourselves from the thread implementations to the thread scalability... ignore the stuff that required for all threads, unless its required for many threads. (I didn't find any implementations that required hardware)<br />
<br />
I would also argue that since OSs have to run on multiple hardwares one cannot guarantee that unique/rare hardware bits will be there. While we can talk about hardware we should limit it to a mention at most. OR we could mention prospective hardware that could help out, but is not yet standard. It depends on whether we want to do "as it is" or "as it might be"<br />
<br />
utility of such massively scalable thread implementations. I took this as: what functionality (of single strings) does one have to give up to make threads scalable.<br />
<br />
<br />
'''Gautam:<br>'''<br />
I think the hardware is as relevant as the software. Not all things can be done in software and hardware support is an important factor in most of the solutions to many problems that OS face. My take.<br />
<br />
'''Henry:<br>'''<br />
Since the question is about the system as a whole, I think the answer should include both software and hardware support for large amounts of threads. The questions revolves around how a system can handle millions of threads and what are the major factors that allow the system to do it. Also, the last part of the question seems to ask what this amount of threads allows a process to do.<br />
<br />
<br />
'''Shane:<br>'''<br />
In response to the above's idea on the last part of the question, I would argue that it would enable fast execution because all threads that receive a cache miss would be picked up by the other threads so long as there was enough resources. Also the use of more threads would help synchronize the cache (through sharing) so that it would not miss. Of course this would be if they were assigned to the same task, you cannot sync threads running different applications it just wouldn't make sense. The only issue with this idea is the software must support this number.<br />
<br />
== Group 7 ==<br />
<br />
Let us start out by listing down our names and email id (preffered). <br />
<br />
Gautam Akiwate <gautam.akiwate@gmail.com><br />
<br />
Patrick Young(rannath) <rannath@gmail.com><br />
<br />
vG Vivek <support.tamiltreasure@gmail.com><br />
<br />
Shane Panke <shanepanke@msn.com><br />
<br />
Henry Irving <sens.henry@gmail.com><br />
<br />
Paul Raubic <paul_raubic@hotmail.com><br />
<br />
== Guidelines ==<br />
<br />
Raw info should have some indication of where you got it for citation.<br />
<br />
Claim your info so we don't need to dig for who got what when we need clarification.<br />
<br />
Feel free to provide info for or edit someone else's info, just keep their signature so we can discuss changes<br />
<br />
sign changes (once) preferably without time stamps Ex: --[[User:Rannath|Rannath]]<br />
<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
== Log ==<br />
Please maintain a log of your activities in the Log Section. So that we can keep track of the evolution of the essay. --[[User:Gautam|Gautam]]<br />
<br />
Moved around some info for clarity<br />
<br />
everyone should post your interpretation of the question in simplest possible English so we`re on the same page (as someone, maybe me, seems to have the wrong idea about what we`re trying to talk about) <br />
<br />
More moving for clarity<br />
added an essay outline at bottom (feel free to change)<br />
filled in the outline somewhat<br />
added questions to the outline for everyone to think on.--[[User:Rannath|Rannath]]<br />
<br />
First Draft for essay. Please modify and add on. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
<br><Add your future activities here><br />
<br />
== Facts We have ==<br />
Start by placing the info here so we can sort through it. I'm going to go into full research/essay writing mode on Sunday if there isn't enough here.<br />
<br />
So far we have:<br />
Three design choices I've seen:<br />
# Smallest possible footprint per-thread (being extremely light weight) - from everywhere<br />
# least number (none if at all possible) of context switches per-thread - ''5''<br />
# use of a "thread pool" - ''3''<br />
The idea is to reduce processor time and storage needed per-thread so you can have more in the same amount of space. --[[User:Rannath|Rannath]]<br />
<br />
----<br />
These are all related ideas.<br />
<br />
Ok, since we are discussing design choices maybe we could also elaborate on the two major types of threads. Here, I already wrote a few lines, source can be found in citation section: <br />
<br />
''Fibers (user mode threads) provide very quick and efficient switching because there is no need for a system call and kernel is oblivious to a switch - allows for millions of user mode threads. ISSUES: Blocking system calls disables all other fibers.<br />
On the other hand managing threads through the kernel requires context switch (between user and kernel mode) on creation and removal of a thread therefore programs with prodigious number of threads would suffer huge performance hits.--[[User:Praubic|Praubic]] 18:05, 10 October 2010 (UTC)''<br />
<br />
<br />
User-mode scheduling (UMS) is a light-weight mechanism that applications can use to schedule their own threads. The ability to switch between threads in user mode makes UMS more efficient than thread pools for short-duration work items that require few system calls. [[Paul]]<br />
<br />
One implementation of UMS is: combination of N:N and N:M, where the N:N relationship reveals N false processors to the user-space so the user can deal with scheduling on their own. ''5'' -[[Rannath]]<br />
<br />
----<br />
<br />
I would scrap the first two below, at most mention them...<br />
<br />
#time-division multiplexing<br />
#threads vs processes<br />
#I/O Scheduling -[[vG]]<br />
<br />
<br />
<br />
<br />
Splitting this off because I don't think it's technically part of the answer<br><br />
Multithreading generally occurs by time-division multiplexing. It makes it possible for the processor to switch between different threads but it happens so fast that the user sees it as it is running at the same time. [[User:vG]]<br />
<br />
<br />
<br />
----<br />
Things that we '''need''' to cover in the essay:--[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br><br />
This is a '''need''' section 4 below is not '''needed'''<br><br />
(A)Design Decisions <br />
1. Type of threading (1:1 1:N M:N)<br />
2. Signal handling - we might be able to leave this out as it seems some "light weight" threads use no signals<br />
3. Synchronisation<br />
4. Memory Handling<br />
5. Scheduling Priorities (context switching and how it affects the CPU threading process)[[Paul]]<br />
----<br />
<br />
Things we might want also to cover in the essay (non-essentials here): --[[User:Rannath|Rannath]] 04:43, 10 October 2010 (UTC)<br><br />
(A)Design Decisions <br />
1. Brief History of threading<br />
2. examples of attempts at getting absurd numbers of threads (failures)<br />
3. other types of threading, including heavy weight and processes<br />
4. Examples of systems that require many threads such as mainframe servers or banking client processing.--[[User:Praubic|Praubic]] 17:34, 11 October 2010 (UTC)<br />
<br />
Here is an example of a design: (the topic asks for key design choices here is one)<br />
<br />
Capriccio is a specific design for scalable user level threads. They are distinct from most designs by being independent of event based mechanisms as well as kernel thread models. They are very good choice for internet servers and this implementations could easily support 100,000 threads. They are characterized by high scalability, efficient stack management and scheduling based on resource usage however the performance is not comparable to event-based systems.--[[User:Praubic|Praubic]] 13:32, 12 October 2010 (UTC)<br />
<br />
(B)Kernel <br />
1. Program Thread manipulation through system calls --[[User:Hirving|Hirving]] 20:05, 7 October 2010 (UTC)<br />
<br />
(C)Hardware --[[User:Hirving|Hirving]] 19:55, 7 October 2010 (UTC)<br />
1. Simultaneous Multithreading<br />
2. Multi-core processors<br />
<br />
== Essay Outline ==<br />
<br />
#Thesis is an answer to the question so... that's the first step, or the last step, we can always present our info and make our thesis match the info.<br />
#List all questions and points we have about the topic<br />
<br />
Questions:<br />
#What makes threads non-scalable? List the problems<br />
#What utility do some scalable implementations lack? Why?<br />
#Just how scalable does a full utility implementation get?<br />
<br />
Answers:<br />
# <br />
# Signals, portability(maybe) both add overhead which would slow down threads<br />
#<br />
----<br />
<br />
Intro (fill in info)<br />
# Thesis<br />
# main topics <br />
<br />
<br />
----<br />
<br />
Body (made of many main points)<br />
<br />
Main Point 1 -[[Rannath]]<br><br />
- efficient thread creation/destruction is more scalable<br><br />
-- NPTL's improvements over LinuxThreads- primarily due to lower overhead of creation/destruction ''1''<br />
<br />
Main Point 2 -[[Rannath]]<br><br />
- UMS & user-space threads are more scalable - maybe<br><br />
-- context switches are costly ''From class''<br><br />
-- blocking locks have lower latency when twinned with a user space scheduler ''8''<br />
<br />
Main Point 3<br><br />
- Certain bottleneck appear in scaled implementations, removing these improves scalability.<br><br />
-- "False cache-line sharing" ''14''<br><br />
-- xtime lock to a lockless lock ''14''<br />
<br />
Main Point 3.5<br><br />
Fine-Grain over course-grain<br><br />
-- "Big Kernel Lock" ''14''<br><br />
-- dcache_lock ''14''<br />
<br />
Link the Main points to the thesis<br />
<br />
----<br />
<br />
Conclusion<br />
# restate info<br />
# affirmation of thesis<br />
<br />
Here is the first paragraph that I attempted. Please feel free to change or even delete it from here. <br />
<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable User threads on the other hand are mapped to kernel threads by the threads library such as libpthreads. and there are a few designs that incorporate it mainly Fibers and UMS (User Mode Scheudling) which allow for very high scalability. UMS threads have their own context and resources however the ability to switch in the user mode makes them more efficient (depending on application) than Thread Pools which are yet another mechanism that allows for high scalability.<br />
--[[User:Praubic|Praubic]] 19:04, 12 October 2010 (UTC)<br />
<br />
<br />
----<br />
I suggest that we start filling out the main points of the essay. We can discuss the intricacies as we go along. --[[User:Gautam|Gautam]] 02:46, 13 October 2010 (UTC)<br />
<br />
== Sources ==<br />
<br />
# Short history of threads in Linux and new implementation of them. [http://www.drdobbs.com/open-source/184406204;jsessionid=3MRSO5YMO1QVRQE1GHRSKHWATMY32JVN NPTL: The New Implementation of Threads for Linux ] [[User:Gautam|Gautam]] 22:18, 5 October 2010 (UTC)<br />
# This paper discusses the design choices [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.6590&rep=rep1&type=pdf Native POSIX Threads] [[User:Gautam|Gautam]] 22:11, 5 October 2010 (UTC)<br />
# lightweight threads vs kernel threads [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9043&rep=rep1&type=pdf PicoThreads: Lightweight Threads in Java] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# [http://eigenclass.org/http://homeostasis.scs.carleton.ca/wiki/index.php?title=Talk:COMP_3000_Essay_1_2010_Question_7&action=edit&section=7hiki/lightweight-threads-with-lwt Eigenclass Comparing lightweight threads] --[[User:Rannath|Rannath]] 00:23, 6 October 2010 (UTC)<br />
# A lightwight thread implementation for Unix [http://www.usenix.org/publications/library/proceedings/sa92/stein.pdf Implementing light weight threads] --[[User:Rannath|Rannath]] 00:49, 6 October 2010 (UTC) [[User:Gbint|Gbint]] 19:50, 5 October 2010 (UTC)<br />
#Not in this group, but I thought that this paper was excellent: [http://www.sandia.gov/~rcmurph/doc/qt_paper.pdf Qthreads: An API for Programming with Millions of Lightweight Threads]<br />
# Difference between single and multi threading [http://wiki.answers.com/Q/Single_threaded_Process_and_Multi-threaded_Process] [[vG]]<br />
# [http://hdl.handle.net/1853/6804 Implementation of Scalable Blocking Locks using an Adaptative Thread Scheduler] --[[User:Gautam|Gautam]] 19:35, 7 October 2010 (UTC)<br />
# Research Group working on Simultaneous Multithreading [http://www.cs.washington.edu/research/smt/ Simultaneous Multithreading] --[[User:Hirving|Hirving]] 19:58, 7 October 2010 (UTC)<br />
# This site provides in-depth info about threads, threads-pooling, scheduling: http://msdn.microsoft.com/en-us/library/ms684841(VS.85).aspx [[Paul]]<br />
# Here is another site that outlines THREAD designs and techniques: http://people.csail.mit.edu/rinard/osnotes/h2.html [[Paul]]<br />
# [http://www.cosc.brocku.ca/Offerings/4P13/slides/threads.ppt Interesting presentation: really worth checking out] [[Paul]]<br />
# KERNEL vs USERMODE http://www.wordiq.com/definition/Thread_(computer_science)--[[User:Praubic|Praubic]] 18:06, 10 October 2010 (UTC)<br />
# [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.7621&rep=rep1&type=pdf#page=83 Scalability in linux]<br />
# [http://hillside.net/plop/2007/papers/PLoP2007_Ahluwalia.pdf This has something to do with our question...]<br />
# [http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx Scheduling Priorities (Windows)], Microsoft (23 September 2010) --[[User:Spanke|Shane]]<br />
# [http://www.novell.com/coolsolutions/feature/14878.html Linux Scheduling Priorities Explained], Novell (11 October 2005) --[[User:Spanke|Shane]]<br />
# [http://www.ibm.com/developerworks/linux/library/l-completely-fair-scheduler/ Inside the Linux 2.6 Completely Fair Scheduler], IBM (15 December 2009) --[[User:Spanke|Shane]]</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3321COMP 3000 Essay 1 2010 Question 72010-10-13T19:11:03Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31 for Windows (and a Red-Black Tree used by the CFS (Completely Fair Scheduler) in Linux). All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spankehttps://homeostasis.scs.carleton.ca/wiki/index.php?title=COMP_3000_Essay_1_2010_Question_7&diff=3319COMP 3000 Essay 1 2010 Question 72010-10-13T19:01:37Z<p>Spanke: /* Design Choices */</p>
<hr />
<div>=Question=<br />
<br />
How is it possible for systems to supports millions of threads or more within a single process? What are the key design choices that make such systems work - and how do those choices affect the utility of such massively scalable thread implementations?<br />
<br />
=Answer=<br />
A thread is an independent task that executes in the same address space as other threads within a single process while sharing data synchronously. Threads require less system resources then concurrent cooperating processes and start much easier therefore there may exist millions of them in a single process. The two major types of threads are kernel and user-mode. Kernel threads are usually considered more heavy and designs that involve them are not very scalable. User threads on the other hand, are mapped to kernel threads by the threads library such as libpthreads. There are a few designs that incorporate it, mainly Fibers and UMS (User Mode Scheduling) which allow for very high scalability. UMS threads have their own context and resources. However, the ability to switch in the user mode makes them more efficient (depending on the application) than Thread Pools which are yet another mechanism that allows for high scalability.<br><br />
'''Taken the liberty to add Praubic's tentative first para. No changes made as of yet.'''<br />
<br />
----<br />
<br />
One of the challenges in making an existing code base scalable is the identification and elimination of bottlenecks. When porting Linux to a 64-core NUMA system Ray Bryant and John Hawkes found the following bottlenecks (or just wrote a paper about them):<br />
<br />
There can be some instances of misplaced information in the cache that can cause a "cache-coherency operation" to be called. This operation is comparatively expensive. Once misplaced information that causes this problem is identified it can be moved to limit the problem.<br />
<br />
There can also be some user-called locks that contribute to bottlenecks. One such lock is the xtime_lock in Linux. Having locking reading prevented writing to the timer value, leading to starvation. This problem was solved by using a lockless-read.<br />
<br />
The multiqueue scheduler is the third major bottle neck. Altogether, the multiqueue scheduler ate up 25% of the CPU time. It had two problems. The spinlock ate up a fair majority of the CPU time. Whilem, the rest went into computing and recomputing information in the cache. These problems were fixed by replacing the scheduler,. The scheduler was then replaced by a more efficient scheduler [O(1) scheduler].<br />
<br />
The next few bottle necks are related. They're both examples of course-granularity locks eating CPU time. Granularity refers to the execution time of a code segment. The closer a segment is to the speed of an atomic action the finer its granularity.<br />
<br />
One big course-grained bottleneck in the system is the "Big Kernel Lock" (BKL) linux's kernel synchronization control. Waiting for the BKL took up as much as 70% of the CPU time on a system with only 28 cores. The preferred method, on Linux NUMA systems, was to limit the BKL's usage. The ext2 and ext3 file systems were replaced with a file system that uses finer-grained locking (XFS), reducing the impact of the bottle necks.<br />
<br />
The last course-grained bottleneck was the dcache_lock. It ate up a adequate amount of time in normal use. But it was also called in the much more popular dnotify_parent() function, which made it unacceptable. So the dcache_lock strategy was replaced with a finer-grained strategy from a later implementation of linux.<br />
<br />
<br />
== Design Choices ==<br />
'''(A) Kernel Threads and User Threads (1:1 vs M:N)<br>'''<br />
This is the most basic design choice. The 1:1 boasts of a slim clean library interface on top of the kernel functions. Although, the M:N would implement a complicated library, it would offer advantages in areas of signal handling. A general consensus was that the M:N design was not compatible with the Linux kernel due to such a high cost for implementation. This gave birth to the 1:1 model.<br />
'''<br>(B)Signal Handling<br>'''<br />
The kernel implements the POSIX signal handling for use with the multitude of signal masks. Since the signal will only be sent to a thread if it is unblocked, no unnecessary interruptions through signals occur. The kernel is also in a much better situation to judge which is the best thread to receive the signal. This only holds true if the 1-on-1 model is used.<br />
'''<br>(C)Synchronization<br>'''<br />
The implementation of the synchronization primitives such as mutexes, read-write locks, conditional variables, semaphores, and barriers requires some form of kernel support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application. Fortunately, new functionality was added to the kernel to implement all kinds of synchronization.<br />
'''<br>(D)Memory Management<br>'''<br />
One of the goals for the library is to have low startup costs for threads so that scalability is possible. The biggest problem, time-wise, outside the kernel is the memory needed for the thread data structures, thread-local storage, and the stack. This is corrected by optimizing the memory allocation for the threads.<br />
'''<Working on this section> [[hirving]]'''<br />
'''<br>(E)Scheduling Priorities<br>'''<br />
A thread is an entity that can be scheduled according to its scheduling priority which is a number ranging from 0 to 31. All threads are executed in a time splice assigned to them in round robin fashion and lower priority threads wait until the ones above finish performing their tasks. Threads are composed of thread context which internally breaks down into set of machine registers, the kernel and user stack all linked to the address space of the process where the thread resides. A context switch occurs as the time splice elapses and an equal (or higher) priority thread becomes available and it is responsible for allowing high scalability if it is efficiently implemented. For example fibers which are executed entirely in userspace do not require a system call during a switch which highly increases efficiency. --[[User:Praubic|Praubic]] 18:24, 13 October 2010 (UTC)<br />
<br />
== References ==</div>Spanke