Soma-notes - User contributions [en]

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T23:45:31Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*contribution (conclusion mostly): just need to re-work some sections and follow the cues I left in the Conclusion section.
*critique (conclusion mostly): critique the conclusion of the essay
*style: the style section is largely untouched. Daniel and I([[Rannath]]) have puts some thoughts there, but that section needs to be made into sentences.

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====One paragraph====
Through this research on various techniques as listed above, it was determined by the authors that the Linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux's system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the Linux community has improved Linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to change how the applications are programmed rather than make changes to the Linux kernel in order to make scalability improvements. This may indicate that what has come before is done quite well when considering that the Linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

====The other====
The contribution of this paper is a lot of research that has focus upon techniques and methods for scalability. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some beneficial aspects. Some of the sections listed provide insight into the improvements that can be reaped from these optimizations.

*Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.
**Or you could change your paragraphs to be more inline with everyone else's work.
**Or since I still don't know what you want to can put what needs to change where it needs to change as a cue.
*** I want to write contribution in the sense that it answers the questions posed. How does this contribute and does it improve upon previous iterations. The idea is that the research contributes, the sections back it up and the the previous iterations actually show merit as effective techniques used for scalability.
****Each and every one removes a bottleneck, that's how they improve it, That's right at the top after that I just listed the bottlenecks removed.
***If you want to state it in a matter of fact, the contribution is in my opinion the research itself which shows that rather than going in and fixing all the bottlenecks discovered that the research showed that application programming can be greatly optimized instead of the kernel. That's the contribution it added, not that they went in and fixed all the bottlenecks.

====What needs to happen====
*Rovic your bits was formed as a conclusion, built from the essay, not a grounds to build our arguments on. So I moved it here from the header. Your paragraph is right, you just had it in the wrong place. -[[Rannath]]
*Both the above sections need to be joined in such a way as to eliminate repeated information.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T22:55:27Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*contribution (conclusion mostly): just need to re-work some sections and follow the cues I left in the Conclusion section.
*critique (conclusion mostly): critique the conclusion of the essay
*style: the style section is largely untouched. Daniel and I([[Rannath]]) have puts some thoughts there, but that section needs to be made into sentences.

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====One paragraph====
Through this research on various techniques as listed above, it was determined by the authors that the Linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux's system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the Linux community has improved Linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to change how the applications are programmed rather than make changes to the Linux kernel in order to make scalability improvements. This may indicate that what has come before is done quite well when considering that the Linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

====The other====
The contribution of this paper is a lot of research that has focus upon techniques and methods for scalability. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some beneficial aspects. Some of the sections listed provide insight into the improvements that can be reaped from these optimizations.

*Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.
**Or you could change your paragraphs to be more inline with everyone else's work.
**Or since I still don't know what you want to can put what needs to change where it needs to change as a cue.
*** I want to write contribution in the sense that it answers the questions posed. How does this contribute and does it improve upon previous iterations. The idea is that the research contributes, the sections back it up and the the previous iterations actually show merit as effective techniques used for scalability.

====What needs to happen====
*Rovic your bits was formed as a conclusion, built from the essay, not a grounds to build our arguments on. So I moved it here from the header. Your paragraph is right, you just had it in the wrong place. -[[Rannath]]
*Both the above sections need to be joined in such a way as to eliminate repeated information.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:54:57Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects. Some of the sections listed provide insight into the improvements that can be reaped from these optimizations.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements. This can indicate that what has come before is done quite well when considering that the linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:54:09Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements. This can indicate that what has come before is done quite well when considering that the linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:53:57Z

Slade: /* What hinders scalability: Section 4.1 */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements. This can indicate that what has come before is done quite well when considering that the linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:53:12Z

Slade: /* Rovic P. */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects.

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements. This can indicate that what has come before is done quite well when considering that the linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-Hey guys, if anyone can see this, if you have a chance, expand on the sections from 4.2 to 4.7 to focus on the contribution analysis I've written. I'm a bit sleep deprived to proof anything I write. I will have more of a chance to add to it after 8pm today.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:49:33Z

Slade: /* Avoiding unnecessary locking: Section 4.7 */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects.

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements. This can indicate that what has come before is done quite well when considering that the linux kernel optimizations showed more improvement when put in conjunction with the application improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:47:29Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects.

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements.

==============================================================================================================
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T20:46:51Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper - DONE!!!==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

The paper: [http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

==Background Concepts - DONE!!!==
===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance of memcached running on many different cores is bottlenecked by an internal lock, which is avoided by the MIT team by running one instance per core. Clients each connect to a single instance of memcached, allowing the server to simulate parallelism without needing to make major changes to the application or kernel. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.[1]

===Apache: ''Section 3.3''===
Apache is a web server that has been used in previous Linux scalability studies. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (making it a perfect example of parallel programming). Each process uses one of their threads to accepting incoming connections and others are used to process these connections. On a single core processor, Apache spends 60% of its execution time in the kernel.[1]

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem - DONE!!!==
As technology progresses, the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[[#Foot1|1]]. The problem with a standard Linux OS is that they are not designed for massive scalability, which will soon prove to be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic states that the situation makes sense because there are 48 cores dividing the work, the information should be processed as fast as possible with each core doing as much work as possible.

To fix those scalability issues, it is necessary to focus on three major areas: the Linux kernel, user level design and how applications use kernel services. The Linux kernel can be improved by optimizing sharing and use the current advantages of recent improvement to scalability features. On the user level design, applications can be improved so that there is more focus on parallelism since some programs have not implemented those improved features. The final aspect of improving scalability is how an application uses kernel services to better share resources so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found easily and actually only take simple changes to correct or avoid.[[#Foot1|1]]

This research uses a foundation of previous research discovered during the development of scalability in UNIX systems. The major developments from shared memory machines[[#Foot2|2]] and wait-free synchronization to fast message passing ended up creating a base set of techniques, which can be used to improve scalability. These techniques have been incorporated in all major operating system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems, such as Read-Copy-Update, which is an algorithm that is used to avoid locks and atomic instructions which affect scalability.[[#Foot3|3]] There is an excellent base of research on Linux scalability studies that have already been written, on which this research paper can model its testing standards. These papers include research on improving scalability on a 32-core machine.[[#Foot4|4]] In addition, the base of studies can be used to improve the results of these experiments by learning from the previous results. This research may also aid in identifying bottlenecks which speed up creating solutions for those problems.

==Contribution==
What was implemented? Why is it any better than what came before?

The research contribution of this paper is a lot of research that has focus upon techniques and methods which focus on scalability improvements. This is accomplished through programming of applications alongside kernel programming. This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. In looking at the issue of scalability it is important to note the causes of the factors which hinder scalability.

===What hinders scalability: ''Section 4.1''===
*The percentage of serialization in a program has a lot to do with how much an application can be sped up. This is Amdahl's Law
** Amdahl's Law states that a parallel program can only be sped up by the inverse of the proportion of the program that cannot be made parallel (e.g. 25%(.25) non-parallel --> limit of 4x speedup) (I can't get this to sound right someone fix it please -[[Rannath]] <- I will fix [[Daniel B.]]
*Types of serializing interactions found in the MOSBENCH apps:
**Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
**Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
**Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
**Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
**Not enough tasks for cores leads to idle cores

It has been shown that simple scaling techniques can be effective in increasing scalability. The authors looked at three different approaches to removing the bottlenecks within the system. The first was to see if there were issues within the linux kernel application, the second was to identify issues with the application design and the third was to address how the application interacts with the linux kernel services. Through this approach, the authors were able to quickly identify problems such as bottlenecks and apply simple techniques in fixing the issues at hand to reap some benefitial aspects.

===Multicore packet processing: ''Section 4.2''===
Linux packet processing technique requires the packets to travel along several queues before it finally becomes available for the application to use. This technique works well for most general socket applications. In recent kernel releases Linux takes advantage of multiple hardware queues (when available on the given network interface) or Receive Packet Steering[1] to direct packet flow onto different cores for processing. Or even go as far as directing packet flow to the core on which the application is running using Receive Flow Steering[2] for even better performance. Linux also attempts to increase performance using a sampling technique where it checks every 20th outgoing packet and directs flow based on its hash. This poses a problem for short lived connections like those associated with Apache since there is great potential for packets to be misdirected.

In general this technique performs poorly when there are numerous open connections spread across multiple cores due to mutex (mutual exclusion) delays and cache misses. In such scenarios its better to process all connections, with associated packets and queues, on one core to avoid said issues. The patched kernel's implementation proposed in this article uses multiple hardware queues (which can be accomplished through Receive Packet Sharing) to direct all packets from a given connection to the same core. In turn Apache is modified to only accept connections if the thread dedicated to processing them is on the same core. If the current core's queue is found to be empty it will attempt to obtain work from queues located on different cores. This configuration is ideal for numerous short connections as all the work for them in accomplished quickly on one core avoiding unnecessary mutex delays associated with packet queues and inter-core cache misses.

[1] J. Corbet. Receive Packet Steering, November 2009. http://lwn.net/Articles/362339/.

[2] J. Edge. Receive Flow Steering, April 2010. http://lwn.net/Articles/382428/.

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

Through this research on various techniques as listed above, it was determined by the authors that the linux kernel itself has many incorporated techniques used to improve scalability. The authors actually go on to speculate that "perhaps it is the case that Linux’s system-call API is well suited to an implementation that avoids unnecessary contention over kernel objects." This tends to show that the work in the linux community has improved linux a large amount and is current with modern techniques for optimization. It could also be interpreted from the paper that it may be to the benefit of the community to rather change how the applications are programmed rather than make changes to the linux kernel in order to make scalability improvements.

**************************************************************************************************************
**************************************************************************************************************
Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

All contributions in this paper are the result of the identification and removal or marginalization of bottlenecks.

===Conclusion: ''Sections 6 & 7''===
====Work in Progress====

=====[[Rovic P.]]=====
-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

=====[[Rannath]]=====
Everything so far indicates that the MOSBENCH applications can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [3]. In the end memcached was bottlenecked by the network card.

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

''Which is not a problem as the paper specifically states that they are testing what they can improve in spite of hardware limitation.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Given that all tests are more or less fair for the purposes of the benchmarks. They would support the Hypothesis that Linux can be made to scale, at least to 48 cores. Thus the conclusion is fair iff the rest of the paper is fair.

Now you just have to fill in how fair the rest of the paper is.

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

Some acronyms aren't explained before they are used, so some people reading the paper may get confused as to what they mean (e.g. Linux TLB). Since this paper is meant to be formal, acronyms should be explained, with some exceptions like OS and IBM. - Daniel B.

Your example has no impact on the paper, it was in the "look here for more info" section. Most people wouldn't know what a "translation look-aside buffer" is either.

==References==

[3] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

'''the paper itself doesn't need to be referenced more than once as this is a critique of the paper...'''
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T16:31:33Z

Slade: /* memchached: Section 3.2 */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)
-- Go to Wireless Lab next to CCSS Lounge. Andrew and Dan B. will be there.

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want. -Rannath

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

- I wont be there either. that does not mean i wont/cant contribute. I'll be on msn or you can just email me. -kirill

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*flesh out the whole lot

===Claim Sections===
* I claim Exim and memcached for background and critique -[[Rannath]]
* also per-core data structures, false sharing and unessesary locking for contribution -[[Rannath]]
* For starters I will take the Scalability Tutorial and gmake. Since the part for gmake is short in the paper, I will grab a few more sections later on. - [[Daniel B.]]
* Also, I will take sloppy counters as well - [[Daniel B.]]
* I'm gonna put some work into the apache and postgresql sections - kirill
* Just as a note Anil in class Thuesday the 30th of November said that we only need to explain 3 of the applications and not all 7 - [[Andrew]]
* I'll do the Research problem and contribution sections. - [[Andrew]]
* I will work on contribution - [[Rovic]]
* I'm gonna whip something up for 4.2 since there appears to be nothing mentioned about it. -kirill

* So here is the claims and unclaimed section. Add your name next to one if you want to take it on.
** gmake - Daniel B.
** memcached - Rannath
** Apache - Kirill
** [[(Exim, PostgreSQL, Metis, and Psearchy will not be needed as the professor said we only need to explain 3)]]
** Research Problem - Andrew
** Contribution - Rovic
** Essay Conclusion (also discussion) - Everyone
** Critic, Style - Everyone
** References - Everyone

=Essay=
==Paper==
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

[http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf The paper: An Analysis of Linux Scalability to Many Cores]

==Background Concepts==
Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

Ideas to explain:
- thread (maybe)
- Linux's move towards scalability precedes this paper. (assert this, no explanation needed, maybe a few examples)
- Summarize scalability tutorial (Section 4.1 of the paper) focus on what makes something (non-)scalable
- Describe the programs tested (what they do, how they're programmed (serial vs parallel), where to the do their processing)

===memcached: ''Section 3.2''===
memcached is an in-memory hash table server. One instance running on many cores is bottlenecked by an internal lock. The MIT team ran one instance per-core to avoid the problem. Clients each connect to a single instance. This allows the server to simulate parallelism. With few requests, memcached spends 80% of its time in the kernel on one core, mostly processing packets.

===Apache: ''Section 3.3''===
Apache is a web server. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (Making it a perfect example of parallel programming). One thread to service incoming connections and various other threads to service those connections. On a single core processor, Apache spends 60% of its execution time in the kernel.

===gmake: ''Section 3.5''===
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake takes a file called a makefile and processes its recipes for the requisite files to determine how and when to remake or recompile code. With a simple command -j or --jobs, gmake can process many of these recipes in parallel. Since gmake creates more processes than cores, it can make proper use of multiple cores to process the recipes.[2] Since gmake involves much reading and writing, in order to prevent bottlenecks due to the filesystem or hardware, the test cases use an in-memory filesystem tmpfs, which gives them a backdoor around the bottlenecks for testing purposes. In addition to this, gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution, which limits its scalability to a small degree. gmake spends much of its execution time with its compiler, processing the recipes and recompiling code, but still spend 7.6% of its time in system time.[1]

[2] http://www.gnu.org/software/make/manual/make.html

==Research problem==
my references are just below because it is easier for numbering the data later.

As technological progress the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[1]. The problem with a standard Linux operating is they are not designed for massive scalability which will soon be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic that situation makes sense because 48 cores are dividing the work. But when processing information a process the main goal is to finish so as long as possible every core should be doing a much work as possible.

To fix those scalability issues it is necessary to focus on three major areas: the Linux kernel, user level design and how application use of kernel services. The Linux kernel can be improved be to improve sharing and have the advantage of recent iterations are beginning to implement scalability features. On the user level design applications can be improved so that there is more focus on parallelism since some programs have not implements those improved features. The final aspect of improving scalability is how an application uses kernel services to share resources better so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found actually only take a little work to avoid.[1]

This research is based on much research which was created before in the development of scalability for UNIX system. The major developments from shared memory machines [2], wait-free synchronization to fast message passing have created a base set of techniques which can be used to improve scalability. These techniques have been incorporated in all major operation system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems such as Read-Copy-Update which an algorithm for which is used to avoid locks and atomic instructions which lower scalability.[3] The is also an excellent base a research on Linux scalability studies to base this research paper. These paper include a on doing scalability on a 32-core machine. [4] That research can improve the results by learning from the experiments already performed by researchers. This research also aid identifying bottlenecks which speed up researching solutions for those bottlenecks.

[2] J. Kuskin, D. Ofelt, M. Heinrich, J. Heinlein, R. Simoni, K. Gharachorloo, J. Chapin, D. Nakahira, J. Baxter, M. Horowitz, A. Gupta, M. Rosenblum, and J. Hennessy. The Stanford FLASH multiprocessor. In Proc. of the 21st ISCA, pages 302–313,1994.

[3] P. E. McKenney, D. Sarma, A. Arcangeli, A. Kleen, O. Krieger, and R. Russell. Read-copy-update. In Proceedings of the Linux Symposium 2002, pages 338-367, Ottawa Ontario, June 2002

[4] C. Yan, Y. Chen, and S. Yuanchun. OSMark: A benchmark suite for understanding parallel scalability of operating systems on large scale multi-cores. In 2009 2nd International Conference on Computer Science and Information Technology, pages 313–317, 2009

==Contribution==
What was implemented? Why is it any better than what came before?

Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-==Work in Progress==-- -Rovic P.
This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

===Section 4.1 problems:===
**The percentage of serialization in a program has a lot to do with how much an application can be sped up. As from the example in the paper, it seems to follow Amdahl's law (e.g. 25% serialization --> limit of 4x speedup).
**Types of serializing interactions found in the MOSBENCH apps:
***Locking of shared data structure as the number of cores increase leads to an increase in lock wait time
***Writing to shared memory as the number of cores increase leads to an increase in the execution time of the cache coherence protocol
***Competing for space in shared hardware cache as the number of cores increase leads to an increase in cache miss rate
***Competing for shared hardware resources as the number of cores increase leads to time lost waiting for resources
***Not enough tasks for cores leads to idle cores

===Multicore packet processing: ''Section 4.2''===

===Sloppy counters: ''Section 4.3''===
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

===Lock-free comparison: ''Section 4.4''===
This section describes a specific instance of unnecessary locking.

===Per-Core Data Structures: ''Section 4.5''===
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

===Eliminating false sharing: ''Section 4.6''===
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

===Avoiding unnecessary locking: ''Section 4.7''===
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

==Conclusion: ''Sections 6 & 7''==
Everything so far indicates that MOSBENCH application can scale to 48 cores. This scaling required a few modest changes to remove bottlenecks. The MIT team speculate that that trend will continue as the number of cores increase. They also state that things not bottlenecked by the CPU are harder to fix.

We can eliminate most kernel bottlenecks that the applications hits most often with minor changes. Most changes were well known methodology, with the exception of Sloppy counters. This study is limited by the removal of the IO bottleneck, but it does suggest that traditional implementations can be made scalable.

==Critique==
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

===Content(Fairness): ''Section 5''===

====memcached: ''Section 5.3''====
memcached is treated with near perfect fairness in the paper. Its an in-memory service, so the ignored storage IO bottleneck does not affect it at all. Likewise the "stock" and "PK" implementations are given the same test suite, so there is no advantage given to either. memcached itself is non-scalable, so the MIT team was forced to run one instance per-core to keep up throughput. The FAQ at memcached.org's wiki suggests using multiple implementations per-server as a work around to another problem, which implies that running multiple instances of the server is the same, or nearly the same, as running one larger server [1]. In the end memcached was bottlenecked by the network card.

[1] memcached's wiki: http://code.google.com/p/memcached/wiki/FAQ#Can_I_use_different_size_caches_across_servers_and_will_memcache

====Apache: ''Section 5.4''====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. The patched kernel implementation of the network stack is also specific to the problem at hand, which is processing multiple short lived connections across multiple cores. Although this provides a performance increase in the given scenario, in more general applications network performance might suffer. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware. ''Which is not a problem as the paper specifically states that there are hardware limitations.'' - [[Rannath]]

====gmake: ''Section 5.6''====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

====Conclusion: ''Sections 6 & 7''====
Fair conclusion?

===Style===
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing? Wrong?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

Everything seems to be in logical order. I couldn't find any needless info. Nothing inherently confusing or wrong. Nothing bad on the grammar front either. - Rannath

==References==
[1] Silas Boyd-Wickizer et al. "An Analysis of Linux Scalability to Many Cores". In ''OSDI '10, 9th USENIX Symposium on OS Design and Implementation'', Vancouver, BC, Canada, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf.

You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

==Deprecated==
===Background Concepts===
* Exim: ''Section 3.1'':
**Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

* PostgreSQL: ''Section 3.4'':
**As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T14:12:20Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want.

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*Background Concepts -fill in info (fii)
*Contribution -fii
*Critique -fii
*References -fii

===Claim Sections===
* I claim Exim and memcached for background and critique -[[Rannath]]
* also per-core data structures, false sharing and unessesary locking for contribution -[[Rannath]]
* For starters I will take the Scalability Tutorial and gmake. Since the part for gmake is short in the paper, I will grab a few more sections later on. - [[Daniel B.]]
* Also, I will take sloppy counters as well - [[Daniel B.]]
* I'm gonna put some work into the apache and postgresql sections - kirill
* Just as a note Anil in class Thuesday the 30th of November said that we only need to explain 3 of the applications and not all 7 - [[Andrew]]
* I'll do the Research problem and contribution sections. - [[Andrew]]
* I will work on contribution - [[Rovic]]

=Essay=
===Paper===
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

[http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf The paper: An Analysis of Linux Scalability to Many Cores]

===Background Concepts===
Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

Ideas to explain:
- thread (maybe)
- Linux's move towards scalability precedes this paper. (assert this, no explanation needed, maybe a few examples)
- Summarize scalability tutorial (Section 4.1 of the paper) focus on what makes something (non-)scalable
- Describe the programs tested (what they do, how they're programmed (serial vs parallel), where to the do their processing)

====Exim: ''Section 3.1''====
Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

====memchached: ''Section 3.2''====
memcached is an in-memory hash table. memchached is very much not parallel, but can be made to be, you just have to run multiple instances. Have clients worry about synchronizing data between the different instances. With few requests memcached does most of its processing at the network stack, 80% of its time on one core.

====Apache: ''Section 3.3''====
Apache is a web server. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (Making it a perfect example of parallel programming). One thread to service incoming connections and various other threads to service those connections. On a single core processor, Apache spends 60% of its execution time in the kernel.

====PostgreSQL: ''Section 3.4''====
As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

====gmake: ''Section 3.5''====
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake is already quite parallel, creating more processes than cores, so that it can make proper use of multiple cores, and involves much reading and writing of files, as it is used to build the Linux kernel. gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution. gmake spends much of its execution time with its compiler, but still spend 7.6% of its time in system time.

====Psearchy: ''Section 3.6''====

====Metis: ''Section 3.7''====

===Research problem===
my references are just below because it is easier for numbering the data later.

As technological progress the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[1]. The problem with a standard Linux operating is they are not designed for massive scalability which will soon be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic that situation makes sense because 48 cores are dividing the work. But when processing information a process the main goal is to finish so as long as possible every core should be doing a much work as possible.

To fix those scalability issues it is necessary to focus on three major areas: the Linux kernel, user level design and how application use of kernel services. The Linux kernel can be improved be to improve sharing and have the advantage of recent iterations are beginning to implement scalability features. On the user level design applications can be improved so that there is more focus on parallelism since some programs have not implements those improved features. The final aspect of improving scalability is how an application uses kernel services to share resources better so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found actually only take a little work to avoid.[1]

This research is based on much research which was created before in the development of scalability for UNIX system. The major developments from shared memory machines [2], wait-free synchronization to fast message passing have created a base set of techniques which can be used to improve scalability. These techniques have been incorporated in all major operation system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems such as Read-Copy-Update which an algorithm for which is used to avoid locks and atomic instructions which lower scalability.[3] The is also an excellent base a research on Linux scalability studies to base this research paper. These paper include a on doing scalability on a 32-core machine. [4] That research can improve the results by learning from the experiments already performed by researchers. This research also aid identifying bottlenecks which speed up researching solutions for those bottlenecks.

[2] J. Kuskin, D. Ofelt, M. Heinrich, J. Heinlein, R. Simoni, K. Gharachorloo, J. Chapin, D. Nakahira, J. Baxter, M. Horowitz, A. Gupta, M. Rosenblum, and J. Hennessy. The Stanford FLASH multiprocessor. In Proc. of the 21st ISCA, pages 302–313,1994.

[3] P. E. McKenney, D. Sarma, A. Arcangeli, A. Kleen, O. Krieger, and R. Russell. Read-copy-update. In Proceedings of the Linux Symposium 2002, pages 338-367, Ottawa Ontario, June 2002

[4] C. Yan, Y. Chen, and S. Yuanchun. OSMark: A benchmark suite for understanding parallel scalability of operating systems on large scale multi-cores. In 2009 2nd International Conference on Computer Science and Information Technology, pages 313–317, 2009

==Section 4.1 problems:==
**The percentage of serialization in a program has a lot to do with how much an application can be sped up. As from the example in the paper, it seems to follow Amdahl's law (e.g. 25% serialization --> limit of 4x speedup).
**Types of serializing interactions found in the MOSBENCH apps:
***Locking of shared data structure - increasing # of cores --> increase in lock wait time
***Writing to shared memory - increasing # of cores --> increase in wait for cache coherence protocol
***Competing for space in shared hardware cache - increasing # of cores --> increase in cache miss rate
***Competing for shared hardware resources - increasing # of cores --> increase in wait for resources
***Not enough tasks for cores --> idle cores

===Contribution===
What was implemented? Why is it any better than what came before?

Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

-I'm just using this as a notepad, do not copy/paste this section, I will put in a properly written set of paragraphs which will fit with the contribution questions asked. -RP

-==Work in Progress==-- -Rovic P.
This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

====Multicore packet processing: ''Section 4.2''====

====Sloppy counters: ''Section 4.3''====
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

====Lock-free comparison: ''Section 4.4''====

====Per-Core Data Structures: ''Section 4.5''====
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

====Eliminating false sharing: ''Section 4.6''====
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

====Avoiding unnecessary locking: ''Section 4.7''====
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

====Conclusion====
Conclusion: we can make a traditional OS architecture scale (at least to 48 cores), we just have to remove bottlenecks.

===Critique===
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

====Content(Fairness): ''Section 5''====
Fairness criterion:
- does the test accurately describe real-world use-cases (or some set there-of)? (external fairness, maybe ignored for testing and benchmarking purposes, usually is too)
- does the test put all tested implementations through the same test? (internal fairness)

Both the stock and new implementations use the same benchmarks, therefore neither of them has a particular advantage. That holds true for all seven programs. In spite of this, there are also some assumptions or conditions that the paper fails to provide a fair explanation as to the inclusion/exclusion of. All tests ignore the storage IO bottleneck, which while not entirely relevant for the purposes of the paper, is relevant to real-world use. It is a problem, but not nearly as bad once you consider SDD technology, which goes a long way to reducing the storage IO bottleneck.

=====memcached: ''Section 5.3''=====
memcached has no explicit or implicit fairness concerns with respect to real-world scenarios.

=====Apache: ''Section 5.4''=====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

=====PostgreSQL: ''Section 5.5''=====

=====gmake: ''Section 5.6''=====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

=====Psearchy: ''Section 5.7''=====

=====Metis: ''Section 5.8''=====

====Style====
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

===References===
You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T13:58:27Z

Slade: /* Contribution */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want.

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*Background Concepts -fill in info (fii)
*Contribution -fii
*Critique -fii
*References -fii

===Claim Sections===
* I claim Exim and memcached for background and critique -[[Rannath]]
* also per-core data structures, false sharing and unessesary locking for contribution -[[Rannath]]
* For starters I will take the Scalability Tutorial and gmake. Since the part for gmake is short in the paper, I will grab a few more sections later on. - [[Daniel B.]]
* Also, I will take sloppy counters as well - [[Daniel B.]]
* I'm gonna put some work into the apache and postgresql sections - kirill
* Just as a note Anil in class Thuesday the 30th of November said that we only need to explain 3 of the applications and not all 7 - [[Andrew]]
* I'll do the Research problem and contribution sections. - [[Andrew]]
* I will work on contribution - [[Rovic]]

=Essay=
===Paper===
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

[http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf The paper: An Analysis of Linux Scalability to Many Cores]

===Background Concepts===
Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

Ideas to explain:
- thread (maybe)
- Linux's move towards scalability precedes this paper. (assert this, no explanation needed, maybe a few examples)
- Summarize scalability tutorial (Section 4.1 of the paper) focus on what makes something (non-)scalable
- Describe the programs tested (what they do, how they're programmed (serial vs parallel), where to the do their processing)

====Exim: ''Section 3.1''====
Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

====memchached: ''Section 3.2''====
memcached is an in-memory hash table. memchached is very much not parallel, but can be made to be, just run multiple instances. Have clients worry about synchronizing data between the different instances. With few requests memcached does most of its processing at the network stack, 80% of its time on one core.

====Apache: ''Section 3.3''====
Apache is a web server. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (Making it a perfect example of parallel programming). One thread to service incoming connections and various other threads to service those connections. On a single core processor, Apache spends 60% of its execution time in the kernel.

====PostgreSQL: ''Section 3.4''====
As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

====gmake: ''Section 3.5''====
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake is already quite parallel, creating more processes than cores, so that it can make proper use of multiple cores, and involves much reading and writing of files, as it is used to build the Linux kernel. gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution. gmake spends much of its execution time with its compiler, but still spend 7.6% of its time in system time.

====Psearchy: ''Section 3.6''====

====Metis: ''Section 3.7''====

===Research problem===
my references are just below because it is easier for numbering the data later.

As technological progress the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[1]. The problem with a standard Linux operating is they are not designed for massive scalability which will soon be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic that situation makes sense because 48 cores are dividing the work. But when processing information a process the main goal is to finish so as long as possible every core should be doing a much work as possible.

To fix those scalability issues it is necessary to focus on three major areas: the Linux kernel, user level design and how application use of kernel services. The Linux kernel can be improved be to improve sharing and have the advantage of recent iterations are beginning to implement scalability features. On the user level design applications can be improved so that there is more focus on parallelism since some programs have not implements those improved features. The final aspect of improving scalability is how an application uses kernel services to share resources better so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found actually only take a little work to avoid.[1]

This research is based on much research which was created before in the development of scalability for UNIX system. The major developments from shared memory machines [2], wait-free synchronization to fast message passing have created a base set of techniques which can be used to improve scalability. These techniques have been incorporated in all major operation system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems such as Read-Copy-Update which an algorithm for which is used to avoid locks and atomic instructions which lower scalability.[3] The is also an excellent base a research on Linux scalability studies to base this research paper. These paper include a on doing scalability on a 32-core machine. [4] That research can improve the results by learning from the experiments already performed by researchers. This research also aid identifying bottlenecks which speed up researching solutions for those bottlenecks.

[2] J. Kuskin, D. Ofelt, M. Heinrich, J. Heinlein, R. Simoni, K. Gharachorloo, J. Chapin, D. Nakahira, J. Baxter, M. Horowitz, A. Gupta, M. Rosenblum, and J. Hennessy. The Stanford FLASH multiprocessor. In Proc. of the 21st ISCA, pages 302–313,1994.

[3] P. E. McKenney, D. Sarma, A. Arcangeli, A. Kleen, O. Krieger, and R. Russell. Read-copy-update. In Proceedings of the Linux Symposium 2002, pages 338-367, Ottawa Ontario, June 2002

[4] C. Yan, Y. Chen, and S. Yuanchun. OSMark: A benchmark suite for understanding parallel scalability of operating systems on large scale multi-cores. In 2009 2nd International Conference on Computer Science and Information Technology, pages 313–317, 2009

==Section 4.1 problems:==
**The percentage of serialization in a program has a lot to do with how much an application can be sped up. As from the example in the paper, it seems to follow Amdahl's law (e.g. 25% serialization --> limit of 4x speedup).
**Types of serializing interactions found in the MOSBENCH apps:
***Locking of shared data structure - increasing # of cores --> increase in lock wait time
***Writing to shared memory - increasing # of cores --> increase in wait for cache coherence protocol
***Competing for space in shared hardware cache - increasing # of cores --> increase in cache miss rate
***Competing for shared hardware resources - increasing # of cores --> increase in wait for resources
***Not enough tasks for cores --> idle cores

===Contribution===
What was implemented? Why is it any better than what came before?

Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

-==Work in Progress==-- -Rovic P.
This research contributes by evaluating the scalability discrepancies of applications programming and kernel programming. Key discoveries in this research show the effectiveness of the kernel in handling scaling amongst CPU cores. This has also shown that scaling in application programming should be more the focus. It has been shown that simple scaling techniques (list techniques) such as programming parallelism (look up more stuff to back this up and quotes). (Sloppy counter effectiveness, possible positive contributions, what has been used (internet search), what hasn’t been used.) Read conclusion, 2nd paragraph.

One reason the
required changes are modest is that stock Linux already
incorporates many modifications to improve scalability.
More speculatively, perhaps it is the case that Linux’s
system-call API is well suited to an implementation that
avoids unnecessary contention over kernel objects.

====Multicore packet processing: ''Section 4.2''====

====Sloppy counters: ''Section 4.3''====
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

====Lock-free comparison: ''Section 4.4''====

====Per-Core Data Structures: ''Section 4.5''====
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

====Eliminating false sharing: ''Section 4.6''====
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

====Avoiding unnecessary locking: ''Section 4.7''====
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

====Conclusion====
Conclusion: we can make a traditional OS architecture scale (at least to 48 cores), we just have to remove bottlenecks.

===Critique===
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

====Content(Fairness): ''Section 5''====
Fairness criterion:
- does the test accurately describe real-world use-cases (or some set there-of)? (external fairness, maybe ignored for testing and benchmarking purposes, usually is too)
- does the test put all tested implementations through the same test? (internal fairness)

Both the stock and new implementations use the same benchmarks, therefore neither of them has a particular advantage. That holds true for all seven programs. In spite of this, there are also some assumptions or conditions that the paper fails to provide a fair explanation as to the inclusion/exclusion of. All tests ignore the storage IO bottleneck, which while not entirely relevant for the purposes of the paper, is relevant to real-world use. It is a problem, but not nearly as bad once you consider SDD technology, which goes a long way to reducing the storage IO bottleneck.

=====Exim: ''Section 5.2''=====
The test uses a relatively small number of connections, but that is also explicitly stated to be a non-issue - "as long as there are enough clients to keep Exim busy, the number of clients has little effect on performance."

This test is explicitly stated to be ignoring the real-world constraint of the IO bottleneck, thus is unfair when compared to real-world scenarios. The purpose was not to test the IO bottleneck. Therefore the unfairness to real-world scenarios is unimportant.

=====memcached: ''Section 5.3''=====
memcached has no explicit or implicit fairness concerns with respect to real-world scenarios.

=====Apache: ''Section 5.4''=====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

=====PostgreSQL: ''Section 5.5''=====

=====gmake: ''Section 5.6''=====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

=====Psearchy: ''Section 5.7''=====

=====Metis: ''Section 5.8''=====

====Style====
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

===References===
You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T13:42:15Z

Slade: /* Class and Notices */

=Class and Notices=
(Nov. 30, 2010) Prof. Anil stated that we should focus on the 3 easiest to understand parts in section 5 and elaborate on them.

- Also, I, Daniel B., work Thursday night, so I will be finishing up as much of my part as I can for the essay before Thursday morning's class, then maybe we can all meet up in a lab in HP and put the finishing touches on the essay. I will be available online Wednesday night from about 6:30pm onwards and will be in the game dev lab or CCSS lounge Wednesday morning from about 11am to 2pm if anyone would like to meet up with me at those times.

- [[I suggest we meet up Thursday morning after Operating Systems in order to discuss and finalize the essay. Maybe we can even designate a lab for the group to meet up in. Any suggestions?]] - Daniel B.

- HP 3115 since there wont be a class in there (as its our tutorial and we know there won't be anyone there)

- If its all the same to you guys mind if I just join you via msn or iirc? Or phone if you really want.

- I'm working today, but I'll be at a computer reading this page/contributing to my section. Depending on how busy I am, I should be able to get some significant writing in before 4pm today on my section and any additional sections required. RP

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

Daniel Sont [mailto:dan.sont@gmail.com dan.sont@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section & add links to supplementary info
*Background Concepts -fill in info (fii)
*Contribution -fii
*Critique -fii
*References -fii

===Claim Sections===
* I claim Exim and memcached for background and critique -[[Rannath]]
* also per-core data structures, false sharing and unessesary locking for contribution -[[Rannath]]
* For starters I will take the Scalability Tutorial and gmake. Since the part for gmake is short in the paper, I will grab a few more sections later on. - [[Daniel B.]]
* Also, I will take sloppy counters as well - [[Daniel B.]]
* I'm gonna put some work into the apache and postgresql sections - kirill
* Just as a note Anil in class Thuesday the 30th of November said that we only need to explain 3 of the applications and not all 7 - [[Andrew]]
* I'll do the Research problem and contribution sections. - [[Andrew]]
* I will work on contribution - [[Rovic]]

=Essay=
===Paper===
This paper was authored by - Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich.

They all work at MIT CSAIL.

[http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf The paper: An Analysis of Linux Scalability to Many Cores]

===Background Concepts===
Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

Ideas to explain:
- thread (maybe)
- Linux's move towards scalability precedes this paper. (assert this, no explanation needed, maybe a few examples)
- Summarize scalability tutorial (Section 4.1 of the paper) focus on what makes something (non-)scalable
- Describe the programs tested (what they do, how they're programmed (serial vs parallel), where to the do their processing)

====Exim: ''Section 3.1''====
Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

====memchached: ''Section 3.2''====
memcached is an in-memory hash table. memchached is very much not parallel, but can be made to be, just run multiple instances. Have clients worry about synchronizing data between the different instances. With few requests memcached does most of its processing at the network stack, 80% of its time on one core.

====Apache: ''Section 3.3''====
Apache is a web server. In the case of this study, Apache has been configured to run a separate process on each core. Each process, in turn, has multiple threads (Making it a perfect example of parallel programming). One thread to service incoming connections and various other threads to service those connections. On a single core processor, Apache spends 60% of its execution time in the kernel.

====PostgreSQL: ''Section 3.4''====
As implied by the name PostgreSQL is a SQL database. PostgreSQL starts a separate process for each connection and uses kernel locking interfaces extensively to provide concurrent access to the database. Due to bottlenecks introduced in its code and in the kernel code, the amount of time PostgreSQL spends in the kernel increases very rapidly with addition of new cores. On a single core system PostgreSQL spends only 1.5% of its time in the kernel. On a 48 core system the execution time in the kernel jumps to 82%.

====gmake: ''Section 3.5''====
gmake is an unofficial default benchmark in the Linux community which is used in this paper to build the Linux kernel. gmake is already quite parallel, creating more processes than cores, so that it can make proper use of multiple cores, and involves much reading and writing of files, as it is used to build the Linux kernel. gmake is limited in scalability due to the serial processes that run at the beginning and end of its execution. gmake spends much of its execution time with its compiler, but still spend 7.6% of its time in system time.

====Psearchy: ''Section 3.6''====

====Metis: ''Section 3.7''====

===Research problem===
my references are just below because it is easier for numbering the data later.

As technological progress the number of core a main processor can have is increasing at an impressive rate. Soon personal computers will have so many cores that scalability will be an issue. There has to be a way that standard user level Linux kernel will scale with a 48-core system[1]. The problem with a standard Linux operating is they are not designed for massive scalability which will soon be a problem. The issue with scalability is that a solo core will perform much more work compared to a single core working with 47 other cores. Although traditional logic that situation makes sense because 48 cores are dividing the work. But when processing information a process the main goal is to finish so as long as possible every core should be doing a much work as possible.

To fix those scalability issues it is necessary to focus on three major areas: the Linux kernel, user level design and how application use of kernel services. The Linux kernel can be improved be to improve sharing and have the advantage of recent iterations are beginning to implement scalability features. On the user level design applications can be improved so that there is more focus on parallelism since some programs have not implements those improved features. The final aspect of improving scalability is how an application uses kernel services to share resources better so that different aspects of the program are not conflicting over the same services. All of the bottlenecks are found actually only take a little work to avoid.[1]

This research is based on much research which was created before in the development of scalability for UNIX system. The major developments from shared memory machines [2], wait-free synchronization to fast message passing have created a base set of techniques which can be used to improve scalability. These techniques have been incorporated in all major operation system including Linux, Mac OS X and Windows. Linux has been improved with kernel subsystems such as Read-Copy-Update which an algorithm for which is used to avoid locks and atomic instructions which lower scalability.[3] The is also an excellent base a research on Linux scalability studies to base this research paper. These paper include a on doing scalability on a 32-core machine. [4] That research can improve the results by learning from the experiments already performed by researchers. This research also aid identifying bottlenecks which speed up researching solutions for those bottlenecks.

[2] J. Kuskin, D. Ofelt, M. Heinrich, J. Heinlein, R. Simoni, K. Gharachorloo, J. Chapin, D. Nakahira, J. Baxter, M. Horowitz, A. Gupta, M. Rosenblum, and J. Hennessy. The Stanford FLASH multiprocessor. In Proc. of the 21st ISCA, pages 302–313,1994.

[3] P. E. McKenney, D. Sarma, A. Arcangeli, A. Kleen, O. Krieger, and R. Russell. Read-copy-update. In Proceedings of the Linux Symposium 2002, pages 338-367, Ottawa Ontario, June 2002

[4] C. Yan, Y. Chen, and S. Yuanchun. OSMark: A benchmark suite for understanding parallel scalability of operating systems on large scale multi-cores. In 2009 2nd International Conference on Computer Science and Information Technology, pages 313–317, 2009

==Section 4.1 problems:==
**The percentage of serialization in a program has a lot to do with how much an application can be sped up. As from the example in the paper, it seems to be an inverse relationship (e.g. 25% serialization --> limit of 4x speedup).
**Types of serializing interactions found in the MOSBENCH apps:
***Locking of shared data structure - increasing # of cores --> increase in lock wait time
***Writing to shared memory - increasing # of cores --> increase in wait for cache coherence protocol
***Competing for space in shared hardware cache - increasing # of cores --> increase in cache miss rate
***Competing for shared hardware resources - increasing # of cores --> increase in wait for resources
***Not enough tasks for cores --> idle cores

===Contribution===
What was implemented? Why is it any better than what came before?

Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?
- So long as we cite the paper and don't pretend the graphs are ours, we are ok, since we are writing an explanation/critic of the paper.

====Multicore packet processing: ''Section 4.2''====

====Sloppy counters: ''Section 4.3''====
Bottlenecks were encountered when the applications undergoing testing were referencing and updating shared counters for multiple cores. The solution in the paper is to use sloppy counters, which gets each core to track its own separate counts of references and uses a central shared counter to keep all counts on track. This is ideal because each core updates its counts by modifying its per-core counter, usually only needing access to its own local cache, cutting down on waiting for locks or serialization. Sloppy counters are also backwards-compatible with existing shared-counter code, making its implementation much easier to accomplish. The main disadvantages of the sloppy counters are that in situations where object de-allocation occurs often, because the de-allocation itself is an expensive operation, and the counters use up space proportional to the number of cores.

====Lock-free comparison: ''Section 4.4''====

====Per-Core Data Structures: ''Section 4.5''====
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

====Eliminating false sharing: ''Section 4.6''====
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

====Avoiding unnecessary locking: ''Section 4.7''====
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

====Conclusion====
Conclusion: we can make a traditional OS architecture scale (at least to 48 cores), we just have to remove bottlenecks.

===Critique===
What is good and not-so-good about this paper? You may discuss both the style and content;
be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

====Content(Fairness): ''Section 5''====
Fairness criterion:
- does the test accurately describe real-world use-cases (or some set there-of)? (external fairness, maybe ignored for testing and benchmarking purposes, usually is too)
- does the test put all tested implementations through the same test? (internal fairness)

Both the stock and new implementations use the same benchmarks, therefore neither of them has a particular advantage. That holds true for all seven programs. In spite of this, there are also some assumptions or conditions that the paper fails to provide a fair explanation as to the inclusion/exclusion of. All tests ignore the storage IO bottleneck, which while not entirely relevant for the purposes of the paper, is relevant to real-world use. It is a problem, but not nearly as bad once you consider SDD technology, which goes a long way to reducing the storage IO bottleneck.

=====Exim: ''Section 5.2''=====
The test uses a relatively small number of connections, but that is also explicitly stated to be a non-issue - "as long as there are enough clients to keep Exim busy, the number of clients has little effect on performance."

This test is explicitly stated to be ignoring the real-world constraint of the IO bottleneck, thus is unfair when compared to real-world scenarios. The purpose was not to test the IO bottleneck. Therefore the unfairness to real-world scenarios is unimportant.

=====memcached: ''Section 5.3''=====
memcached has no explicit or implicit fairness concerns with respect to real-world scenarios.

=====Apache: ''Section 5.4''=====
Linux has a built in kernel flaw where network packets are forced to travel though multiple queues before they arrive at queue where they can be processed by the application. This imposes significant costs on multi-core systems due to queue locking costs. This flaw inherently diminishes the performance of Apache on multi-core system due to multiple threads spread across cores being forced to deal with these mutex (mutual exclusion) costs. For the sake of this experiment Apache had a separate instance on every core listening on different ports which is not a practical real world application but merely an attempt to implement better parallel execution on a traditional kernel. These tests were also rigged to avoid bottlenecks in place by network and file storage hardware. Meaning, making the proposed modifications to the kernel wont necessarily produce the same increase in productivity as described in the article. This is very much evident in the test where performance degrades past 36 cores due to limitation of the networking hardware.

=====PostgreSQL: ''Section 5.5''=====

=====gmake: ''Section 5.6''=====
Since the inherent nature of gmake makes it quite parallel, the testing and updating that were attempted on gmake resulted in essentially the same scalability results for both the stock and modified kernel. The only change that was found was that gmake spent slightly less time at the system level because of the changes that were made to the system's caching. As stated in the paper, the execution time of gmake relies quite heavily on the compiler that is uses with gmake, so depending on which compiler was chosen, gmake could run worse or even slightly better. In any case, there seems to be no fairness concerns when it comes to the scalability testing of gmake as the same application load-out was used for all of the tests.

=====Psearchy: ''Section 5.7''=====

=====Metis: ''Section 5.8''=====

====Style====
Style Criterion (feel free to add I have no idea what should go here):
- does the paper present information out of order?
- does the paper present needless information?
- does the paper have any sections that are inherently confusing?
- is the paper easy to read through, or does it change subjects repeatedly?
- does the paper have too many "long-winded" sentences, making it seem like the authors are just trying to add extra words to make it seem more important? - I think maybe limit this to run-on sentences.
- Check for grammar

===References===
You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

gmake:

[http://www.gnu.org/software/make/manual/make.html gmake Manual]

[http://www.gnu.org/software/make/ gmake Main Page]

Talk:COMP 3000 Essay 2 2010 Question 1

2010-12-02T00:12:20Z

Slade: /* Claim Sections */

Talk:COMP 3000 Essay 2 2010 Question 1

2010-11-23T12:46:07Z

Slade: /* Group members */

=Group members=
Patrick Young [mailto:Rannath@gmail.com Rannath@gmail.com]

Daniel Beimers [mailto:demongyro@gmail.com demongyro@gmail.com]

Andrew Bown [mailto:abown2@connect.carleton.ca abown2@connect.carleton.ca]

Kirill Kashigin [mailto:k.kashigin@gmail.com k.kashigin@gmail.com]

Rovic Perdon [mailto:rperdon@gmail.com rperdon@gmail.com]

=Methodology=
We should probably have our work verified by at least one group member before posting to the actual page

=To Do=
*Improve the grammar/structure of the paper section
*Background Concepts -fill in info (fii)
*Research problem -fii
*Contribution -fii
*Critique -fii
*References -fii

===Claim Sections===
* I claim Exim and memcached for background and critique -[[Rannath]]
* also per-core data structures, false sharing and unessesary locking for contribution -[[Rannath]]

=Essay=
===Paper===
The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

Authors in order presented: Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich

affiliation: MIT CSAIL

[http://www.usenix.org/events/osdi10/tech/full_papers/Boyd-Wickizer.pdf An Analysis of Linux Scalability to Many Cores]

===Background Concepts===
Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

Ideas to explain:
#thread (maybe)
#Linux's move towards scalability precedes this paper. (assert this, no explanation needed, maybe a few examples)
#Summarize scalability tutorial (Section 4.1 of the paper)
#Describe the programs tested (what they do, how they're programmed (serial vs parallel), where to the do their processing)

=====Exim: ''Section 3.1''=====
Exim is a mail server for Unix. It's fairly parallel. The server forks a new process for each connection and twice to deliver each message. It spends 69% of its time in the kernel on a single core.

=====memchached: ''Section 3.2''=====
memcached is an in-memory hash table. memchached is very much not parallel, but can be made to be, just run multiple instances. Have clients worry about synchronizing data between the different instances. With few requests memcached does most of its processing at the network stack, 80% of its time on one core.

===Research problem===
What is the research problem being addressed by the paper? How does this problem relate to past related work?

Problem being addressed: scalability of current generation OS architecture, using Linux as an example. (?)

Summarize related works (Section 2, include links, expand information to have at least a summary of some related work)

===Contribution===
What was implemented? Why is it any better than what came before?

Summarize info from Section 4.2 onwards, maybe put graphs from Section 5 here to provide support for improvements (if that isn't unethical/illegal)?

Conclusion: we can make a traditional OS architecture scale (at least to 48 cores), we just have to remove bottlenecks.

=====Per-Core Data Structures=====
Three centralized data structures were causing bottlenecks - a per-superblock list of open files, vfsmount table, the packet buffers free list. Each data structure was decentralized to per-core versions of itself. In the case of vfsmount the central data structure was maintained, and any per-core misses got written from the central table to the per-core table.

=====Eliminating false sharing=====
Misplaced variables on the cache cause different cores to request the same line to be read and written at the same time often enough to significantly impact performance. By moving the often written variable to another line the bottleneck was removed.

=====Avoiding unnecessary locking=====
Many locks/mutexes have special cases where they don't need to lock. Likewise mutexes can be split from locking the whole data structure to locking a part of it. Both these changes remove or reduce bottlenecks.

===Critique===
What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

Since this is a "my implementation is better then your implementation" paper the "goodness" of content can be impartially determined by its fairness and the honesty of the authors.

Fairness criterion:
#does the test accurately describe real-world use-cases (or some set there-of)? (external fairness, maybe ignored for testing and benchmarking purposes, usually is too)
#does the test put all tested implementations through the same test? (internal fairness)

Style Criterion (feel free to add I have no idea what should go here):
#does the paper present information out of order?
#does the paper present needless information?
#does the paper have any sections that are inherently confusing?

=====Testing Method: ''Section 5''=====
Both the stock and new implementations use the same benchmarks, therefore internal fairness is preserved for all seven programs.

=====Exim: ''Section 5.2''=====
The test uses a relatively small number of connections, but that is also implicitly stated to be a non-issue - "as long as there are enough clients to keep Exim busy, the number of clients has little effect on performance."

This test is explicitly stated to be ignoring the real-world constraint of the IO bottleneck, thus is unfair when compared to real-world scenarios. The purpose was not to test the IO bottleneck. Therefore the unfairness to real-world scenarios is unimportant.

=====memcached: ''Section 5.3''=====
memcached has no explicit or implicit fairness concerns with respect to real-world scenarios.

===References===
You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T17:48:31Z

Slade: /* The Essay */

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

----
==== ====
(Not complete but most of article 9)
Classical Virtualization
* VMMs allow programs in virtual environments to run natively other than resource usage
** Dominant instructions executed directly on cpu
** vmm completely controls system resources
** often need to emulate every native instruction which would severely effect the performance
** sensitive instruction that violate safety and encapsulation
** vmm handles them as priviledged instructions

x86 Virtualization
* virtualization in personal work stations rather than mainframes
** rings that allow isolation between virtual machines
** most privileged in ring 0 and least in ring 3. The operating system runs in ring 0 and user apps in ring 3
*** vmm in ring 0 and vms in lesser privilege rings (1 or 3)
*** guestOS believes its in ring 0
* address space compression, where to run the VMM
** if run using guest address space, guest can find out its virtualized or compromise the isolation
* does not trap all sensitive instructions but can handle them, violates classical virtualization description
* some privileged access fail without faulting
* interrupt virtualization - VMM handles AND guestOS handles
* binary translation - improve performance
* rewriting instructions and trapping before problems arrise

Paravirtualization
* guestOS become exposed to vm information so that the guest is aware that it is virtualized and can make decisions based on this
* allows to avoid problem instructions
* Xen
* guestOS must be modified and is not natively running
**works with the hostOS to run efficiently

VMM types
* hostedVMM - executes in hostOS and uses the drivers and support of the OS
* Stand-aloneVMM - runs directly on hardware and uses it's own drivers and services
* hybridVMM - runs a serviceOS where requests to hardware go through (I/O)

Device Emulation
* implement real hardware in software
* completely virtual device that the guest interacts with
* mapped to physical hardware that handles the interactions but the emulation allows conversion
* allows the vm to be easily migrated between machines as it does not rely on the physical hardware
* allows having multiple vms and simplifies sharing (multiplexing)
* poor performance as the vmm needs to do a lot to virtulize the machine

Paravirtualization
* modified guestOS to cooperate with VMM
* VMM does not have to do everything to handle device drivers
* not everything can be paravirtualized
* proprietary os and device drivers can't be paravirtualized
* still allows an increase in performance
* eventing or callback mechanism
** guestOS modifies interrupt mechs
* modifications are not applicable to all guestOS

Dedicated Devices
* does not virtualize device but assigns directly to guest vm
* uses guest's drivers instead of host
* simplifiest vmm by removing handing of i/o securily
* limited physical devices that can be dedicated
* dificult to migrate vm as it depends on the pairing with this resource
* elims over-head of virtualization and simplicity in vmm
* direct memory access not supported

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

[9]Fisher-Ogden J. 2006. Hardware Support for Efficient Virtualization. University of California, San Diego. http://cseweb.ucsd.edu/~jfisherogden/hardwareVirt.pdf

Not completely sure of the citation style used above.

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay
Aaron .L

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. We are comparing Exokernels to Microkernels and Virtual Machines by looking at how the kernel goes about such management and its connections. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels because as this design shows, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it also adds a layer of complexity within the system. This is less efficient than a real machine as it accesses the hardware indirectly. It can be observed by examining how the exokernel provides low level hardware access and provides custom abstraction to those devices. This is done in order to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extent that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. An operating system could not function without the kernel.

A kernel is the lowest level section of an operating system. Within a system, it has the most privileges. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access. This is also where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes. For example, the kernel could manage the File Systems and complete process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, which is a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a change in the system, the entire kernel would need to be compiled. Therefore, due to the amount of processes within it, it would take an inefficient amount of time. Here, a microkernel becomes practical.

The concept of a microkernel, is to reduce the code within the kernel. The microkernel is only included in the kernel if it would impact the system. There are a variety of ways the system could be affected if a microkernel were to be implemented, for example, there would be increased performance and efficiency. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better -Slade

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. [4] In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS provides management for the VMM as well as allowing physical access to devices, hardware and drivers. [6]

The VM is what contains the OS we are running through virtualization. [6] This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. [6] Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM but the guestOS will execute as its own machine, unaware of this mediator.

There are various ways of implementing hardware virtualization in a system to allow VMs to run. This includes device emulation, paravirtualization and dedicated devices. [9]

In device emulation, the VMM provides a complete virtualization of a device for the guestOS to interact with, in software. [9] The VMM will map this virtualized device to the physical resource and handle any interactions between them. This will usually include converting instructions from the guestOS into instructions that are compatible with the device. [9] Device emulation allows for the VM to be migrated easily to another machine as it is not dependent on the physical devices but on the software emulations instead. [9] It also allows for simpler multiplexing between multiple virtual machines as it can handle sharing though these virtualized devices. [9] A drawback of emulation, however, is poor performance as the VMM must handle every request and convert them to be compatible with the physical device. [9] However, despite its poor performance, emulation is the most common form of virtualization.

Paravirtualization allows for a boost in performance by having the guestOS and the hostOS work together to improve performance. [9] In paravirtualization, the guestOS is not a native OS and must be modified so that it is aware that it is a virtualized system. [9] As the guestOS is aware of this, it can now make better decisions about how it accesses devices. As the guestOS will be able to handle its decisions better, the VMM’s responsibility is reduced as it now does not have to translate between the guestOS and the physical devices. [9] Though the performance boost is a great advantage, there are many disadvantages to this. You can only use paravirtualization if you can implement the modifications to the guestOS. Not everything can be paravirtualized and as such, this limits the cases in which this method can be used. [9] Also, every guestOS must be modified in order to be used in paravirtualization. The modifications will differ in various OS and so, there is also the task of implementing these changes to make a guestOS compatible. [9]

Instead of virtualizing the hardware and mediating between with the VMM, dedicated devices allow mapping directly to the guestOS. [9] In this method, the device will use the guestOS’s drivers instead of the hostOS’s. [9] Using this method allows the guestOS to use the hardware to its full extent without having to deal with the VMM. This simplifies the VMM by eliminating the overhead in virtualizating the hardware and handling the requests to devices. [9] However, there are limited physical resources to be dedicated to a guestOS and this also makes migration difficult as the guestOS is dependent on the physical device. [9]

'''notes'''
- it ended up being quite lengthy. I mainly focused on the device virtualization rather than the architecture of a VM (like x86 virtualization). I'll put up my notes for the paper I found for virtualization. I didn't talk about Xen or VMware though. If any of that is needed, I can try to continue working on it tonight but I have another priority.

-try focusing on the emulation side of VM where emulation's weaknesses vs direct hardware access or custom abstraction that exokernels -Slade

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

I made some edits to the first two paragraphs. I just reworded some of the unclear sentences and some grammatical errors. I'll work on editing more of it after comp 3007. Also when all the parts are up i can go through it and link the paragraphs together so it can be read more like an essay --[[User:Aellebla|Aellebla]] 15:18, 14 October 2010 (UTC)

So far so good, if you find some sentences that are off, go ahead and correct them, just note to us in here that you've made changes. Almost done guys! -Slade

==Potential Test Questions==

Add potential test questions here:

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T13:31:30Z

Slade: /* The Essay */

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. It is how the kernel goes about such management and its connections that we are comparing Exokernels to Microkernels and Virtual Machines. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels since by design, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it adds a layer of complexity within the system less efficient than a real machine as it accesses the hardware indirectly. It can be observed how the exokernel provides low level hardware access and provide custom abstraction to those devices to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extend that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. Without the kernel, an operating system could not function.

A kernel is the lowest level section of an operating system. It has the most privileges of the system. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access and where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes such as the File Systems and the process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a fix for the system, the entire kernel would need to be compiled, and due to the amount of processes within it, it would take an inefficient amount of time. This is where a microkernel becomes practical.

The concept of a microkernel is reduce the code within the kernel, and it is only included in the kernel if it would affect the system in any way, for example for performance and efficiency reasons. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better -Slade

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS is provides management for the VMM as well as physical access to devices, hardware and drivers.

The VM is what contains the OS we are running through virtualization. This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM.

[ I'll need to do some more research on the types of virtualization though before I can discuss that. If anyone has more information on them to put up in the points that would be helpful but I'll get to it right after class tomorrow morning. ]

-try focusing on the emulation side of VM where emulation's weaknesses vs direct hardware access or custom abstraction that exokernels -Slade

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

Potential Test Questions

Add potential test questions here:

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T13:31:02Z

Slade:

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. It is how the kernel goes about such management and its connections that we are comparing Exokernels to Microkernels and Virtual Machines. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels since by design, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it adds a layer of complexity within the system less efficient than a real machine as it accesses the hardware indirectly. It can be observed how the exokernel provides low level hardware access and provide custom abstraction to those devices to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extend that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. Without the kernel, an operating system could not function.

A kernel is the lowest level section of an operating system. It has the most privileges of the system. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access and where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes such as the File Systems and the process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a fix for the system, the entire kernel would need to be compiled, and due to the amount of processes within it, it would take an inefficient amount of time. This is where a microkernel becomes practical.

The concept of a microkernel is reduce the code within the kernel, and it is only included in the kernel if it would affect the system in any way, for example for performance and efficiency reasons. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better -Slade

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS is provides management for the VMM as well as physical access to devices, hardware and drivers.

The VM is what contains the OS we are running through virtualization. This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM.

[ I'll need to do some more research on the types of virtualization though before I can discuss that. If anyone has more information on them to put up in the points that would be helpful but I'll get to it right after class tomorrow morning. ]

-try focusing on the emulation side of VM where emulation's weaknesses vs direct hardware access or custom abstraction that exokernels -Slade

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

Potential Test Questions

Add potential test questions here:

Potential Test Questions

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T13:30:05Z

Slade:

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. It is how the kernel goes about such management and its connections that we are comparing Exokernels to Microkernels and Virtual Machines. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels since by design, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it adds a layer of complexity within the system less efficient than a real machine as it accesses the hardware indirectly. It can be observed how the exokernel provides low level hardware access and provide custom abstraction to those devices to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extend that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. Without the kernel, an operating system could not function.

A kernel is the lowest level section of an operating system. It has the most privileges of the system. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access and where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes such as the File Systems and the process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a fix for the system, the entire kernel would need to be compiled, and due to the amount of processes within it, it would take an inefficient amount of time. This is where a microkernel becomes practical.

The concept of a microkernel is reduce the code within the kernel, and it is only included in the kernel if it would affect the system in any way, for example for performance and efficiency reasons. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better -Slade

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS is provides management for the VMM as well as physical access to devices, hardware and drivers.

The VM is what contains the OS we are running through virtualization. This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM.

[ I'll need to do some more research on the types of virtualization though before I can discuss that. If anyone has more information on them to put up in the points that would be helpful but I'll get to it right after class tomorrow morning. ]

-try focusing on the emulation side of VM where emulation's weaknesses vs direct hardware access or custom abstraction that exokernels -Slade

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

Potential Test Questions

Add potential test questions here:

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T13:01:49Z

Slade: /* The Essay */

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. It is how the kernel goes about such management and its connections that we are comparing Exokernels to Microkernels and Virtual Machines. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels since by design, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it adds a layer of complexity within the system less efficient than a real machine as it accesses the hardware indirectly. It can be observed how the exokernel provides low level hardware access and provide custom abstraction to those devices to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extend that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. Without the kernel, an operating system could not function.

A kernel is the lowest level section of an operating system. It has the most privileges of the system. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access and where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes such as the File Systems and the process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a fix for the system, the entire kernel would need to be compiled, and due to the amount of processes within it, it would take an inefficient amount of time. This is where a microkernel becomes practical.

The concept of a microkernel is reduce the code within the kernel, and it is only included in the kernel if it would affect the system in any way, for example for performance and efficiency reasons. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better -Slade

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS is provides management for the VMM as well as physical access to devices, hardware and drivers.

The VM is what contains the OS we are running through virtualization. This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM.

[ I'll need to do some more research on the types of virtualization though before I can discuss that. If anyone has more information on them to put up in the points that would be helpful but I'll get to it right after class tomorrow morning. ]

-try focusing on the emulation side of VM where emulation's weaknesses vs direct hardware access or custom abstraction that exokernels -Slade

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T12:58:14Z

Slade: /* The Essay */

== Microkernel ==
* Moving kernel functionality into processes contained in user space, e.g. file systems, drivers
* Keep basic functionality in kernel to handle sharing of resources
* Separation allows for manageability and security, corruption in one does not necessarily cause failure in system
* Large amount of moving from a process to Kernel to user space and back again, this is a costly operation.

----

''' Microkernel '''
* try's to minimize the amount of software that is mandatory or required [7]
advantages of Microkernel
* favors a modular system structure [7]
* one failure of a program does not impact any other programs [7]
* can support more than one api or strategies since all programs are separated [7]
==== Microkernel Concepts ====
* piece of code is allowed in the kernel only if moving it outside the kernel would adversely affect the system. [7]
* any subsystem program created must be independent of all other subsystem's, any subsystem that is used can guarantee this from all other subsystems [7]
===== Address Space =====
* a mapping that relates the physical page to the virtual page. [7]
* processor specific [7]
* hide's the hardware's concept of address space [7]
* based off the idea of recursion each subsystem has it's own address space [7]
* the micro kernel provides 3 operations [7]
** Grant [7]
*** allows the owner to give a page to a recipient, provided the recipient want's it the page is removed from the owner's address space and but in the recipients. [7]
*** must be available to the owner. [7]
** Map [7]
*** allows the user to share a page with a recipient [7]
*** page is not removed from the owner's address space. [7]
** Flush [7]
*** remove's the page from all recipients address space [7]
*** how does this work with Grant --[[User:Asoknack|Asoknack]] 19:10, 12 October 2010 (UTC)
* allows memory management and paging out side the kernel
* Map and flush is required for memory manger's and pagers [7]
* can be used to implement access right's [7]
* controlling I/O Right's and driver's are not done at kernel level [7]

===== Thread's IPC =====
* Threads
** in the kernel [7]
** Since a thread has an address space , all changes to the thread need to be done by the kernel [7]
* IPC [7]
** in the kernel IPC
** grant and map also need IPC (So buye the priciple above this has to be in the kernel)[7]
** basic way for sub process to communicate. [7]
* Interrupts
** partially in the kernel [7]
** hard ware is a set of thread's which are empty except for there unique sender id [7]
** transformation of the message to the interrupt is done in the kernel [7]
** the kernel is not involved in device - specific interrupt's and does not understand the interrupt. [7]
*** resting the interrupt is done at user level [7]
** if a privileged command is need it is done implicitly the next time an IPC command is sent from the device [7]

===== Unique Identifiers =====

== Virtual Machine ==
* Partitioning or virtualizing resources among OS virtualization running on top of host OS
* Virtualized OS believe running on full machine on its own

----
System Level Virtualization

=== VMM ===
* stands for Virtual Machine Monitor, also known as the hyper-visor[4]
* responsible for virtualization of hardware(mapping physical to virtual) and the VM that run on top of the virtuallized hardware [4]
* usually a small os with no drivers , so it is coupled with a linux distro that provides device / hardware access [4]
** the os that the VMM is using for driver's is called the hostOS [6]
*the hostOS provides login and physical access to the hardware as well as management for the VMM [6]
=== VM ===
* the OS that the vm is running is called the guestOS [6]
* the guestOS only sees resources that have been allocated to the VM [6]
==== three approaches ====
*Type I virtualization [5]
** runs off the physical hardware [4]
** Isolation of the guestOs from the hardware is done threw processe level protection meachnism[6]
*** ring 0 = VMM [6]
*** ring 1 = VM [6]
*** this means all instructions from the VM must go threw the VMM [6]
** since there can be multiple VM's on a computer the scheduling is done by the VMM [6]
** on boot the VMM creates a hardware platform for the VM [6]
** load's the VM kernel into virtual memory and then boot's it like a regular computer [6]
** ex. Xen [4]
*Type II virtualization [5]
** run off the host Os [4]
** ex. VMware , QEMU [4]
* Para-virtualization [6]
** Similar to Type but use the HostOs for Device driver access [6]
** Provide a virtualization that is similar to hardware [From the paper posted, no citation yet]
** GuestOS and Hypervisor work together to improve performance

== Exokernel ==
* Micro-kernel architecture with limited abstractions, ask for resource, get resource not resource abstraction
* Less functionality provided by kernel, security and handling of resource sharing
* Once application receives resource, it can use it as it wishes/in control
* Keep the basic kernel to handle allocating resources and sharing rather than developing straight to the hardware

----
* multiplex resources securely providing protection to mutual distrustful application threw the use of secure binding's[1]
* Goal of the exokernel is to give LibOS maximum freedom with out allowing them to interfere with each other. to do this the exokernel separates protection from management in doing this it provide 3 important tasks[1]
** tracking ownership of resources [1]
** ensuring protection by guarding all resource usage and binding points (not to shure what binding points are)[1]
** revoking access to the resources [1]
* LibrayOS (LibOs)
** Reduces the number of kernel crossings[1]
** Not trusted by the exokernel so can be trusted by the application , Example given is a bad parameter passed to the LibOs only the application is affected.[1] (So LibOs cant interact with kernel ???)
** Any application running on the Exokernel can change the LibrayOs freely [1]
** Application that use LibOS that implement standard interfaces (POSIX) will be portable on any system with the same interface [1]
** LibOs can be made portable if it is designed to interact with a low-level machine independent level to hide hardware details [1]

=== Exokernel Design ===
==== Design Principles ====
*Securely Expose Hardware [1]
** an Exokernel tries to create low level primitives that the hardware resources can be accessed from, this also includes interrupts,exceptions [1]
** the exokernel also export privileged instructions to the LibOS so that traditional OS abstractions can be implemented (eg Process , address pace)[1]
** Exokernels should avoid resource management except when required protection ( allocation , revocation , ownership)[1]
** application based resource management is the best way to build flexible efficient flexible systems [1]
*Expose allocation[1]
** allow LibOs to request physical resources [1]
** resource allocation should not be automatic, the LibOS should participate in every single allocation decision [1]
*Expose Names[1]
** Use physical name's when ever possible[3] (not to sure what physical names are, I think it is as simple as what the hardware is called)--[[User:Asoknack|Asoknack]] 20:27, 9 October 2010 (UTC)
** Physical names capture useful information [3]
*** safer than and less resource intensive than virtual names as no translations are needed[3]
*Expose Revocation [1]
** use visible revocation protocol [1]
** allows well behaved LibOS to preform application level resource management [1]
** Visible revocation allows the LibOS to choose what instance of the resource to release[1](Visible means that when revocation happens the exokernel tell the LibOS that resource is being revoked)
''' Policy '''
* LibOS handle resource policy decisions
* Exokernels have a policy to decided between competing LibOS (Priority , share of resources)
** it enforces this threw allocation and deallocation (every thing can achieved threw this even what block to write and such)

==== Secure Bindings ====
* Used by the exokernel to allow the LibOS to bind to resources [1]
* Allows the separation of protection and resource use [1]
* only checks authorization during bind time [1]
** Application's with complex needs for resources only authorized during bind.[1]
* access checking is done during access time and there is no need to understand complex resources needs during access[1]
** (this means that the exokernel checks once to make sure an application has authorization once approved, when the application tries to use the resource the exokernel is only concerned about policy conflict's)--[[User:Asoknack|Asoknack]] 18:20, 9 October 2010 (UTC)
** allows the kernel to protect the resources with out understanding what the resource is [1]
*three way's to implement
* Hardware Mechanisms [1]
* Software caching [1]
* Downloading application code [1]
''' Downloading Code to the Kernel '''
* used to implement secure bindings , and improve performance[1]
** eliminate the number of kernel crossings [1]
** downloaded code can be run with out the application to be scheduled [2]
==== Visible Resource Revocation ====
* Used for most resources [1]
** allows for LibOS to help with deallocation [1]
** LibOS are able to garner what resources are scare [1]
* Slower than Invisible as application involvement is required [1]
** ex of when invisible is used is Processor addressing-context identifiers [1]
==== Abort Protocol ====
* allows the exokernel to take resources away from the LibOS [1]
* used when the LibOS fails to respond to the revocation request [1]
* Exokernel must be careful not to delete as the LibOS might need to write some system critical data to the resource [1]

== Comparisons ==
====Exokernel/Microkernel====
'''Similarities'''
* Limited functionality in kernel
** functionality in kernel to handle sharing of resources and security
** avoids programming directly to hardware which creates a dependency
* Additional functionality provided in user space as processes
'''Differences'''
* Minimal abstractions provided by the kernel
** Applications given more power in exokernel

====Exokernel/VM====
'''Similarities'''
* Idea of partitioning resources between applications/OSs
* "Control" of resource given
* Isolation from other applications/OSs
'''Differences'''
* Exokernel runs applications, VM runs OS
* VM uses a hostOS and guestOSs run on top
* Virtualization on VMs, Exokernel deals with real resources
* VM hides a lot of information because it emulates. Exokernel does not.

====Microkernel/VM====
'''Differences'''
* With a virtual machine, you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System.
* This can be costly but the benefits are that it's easier and all the standard OS features are available.

== References ==
[1]<nowiki> Engler, D. R., Kaashoek, M. F., and O'Toole, J. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 251-266. DOI= http://doi.acm.org/10.1145/224056.224076 </nowiki>

[2]<nowiki>Engler, Dawson R. "The Exokernel Operating System Architecture." Diss. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998. Web. 9 Oct. 2010. <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.5054&rep=rep1&type=pdf>.</nowiki>

[3]<nowiki>Kaashoek, M. F., Engler, D. R., Ganger, G. R., Briceño, H. M., Hunt, R., Mazières, D., Pinckney, T., Grimm, R., Jannotti, J., and Mackenzie, K. 1997. Application performance and flexibility on exokernel systems. In Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles (Saint Malo, France, October 05 - 08, 1997). W. M. Waite, Ed. SOSP '97. ACM, New York, NY, 52-65. DOI= http://doi.acm.org/10.1145/268998.266644 </nowiki>

[4]<nowiki>Vallee, G.; Naughton, T.; Engelmann, C.; Hong Ong; Scott, S.L.; , "System-Level Virtualization for High Performance Computing," Parallel, Distributed and Network-Based Processing, 2008. PDP 2008. 16th Euromicro Conference on , vol., no., pp.636-643, 13-15 Feb. 2008
DOI= http://doi.acm.org/10.1109/PDP.2008.85 </nowiki>

[5]<nowiki>Goldberg, R. P. 1973. Architecture of virtual machines. In Proceedings of the Workshop on Virtual Computer Systems (Cambridge, Massachusetts, United States, March 26 - 27, 1973). ACM, New York, NY, 74-112. DOI= http://doi.acm.org/10.1145/800122.803950 </nowiki>

[6]<nowiki>Vallee, G., Naughton, T., and Scott, S. L. 2007. System management software for virtual environments. In Proceedings of the 4th international Conference on Computing Frontiers (Ischia, Italy, May 07 - 09, 2007). CF '07. ACM, New York, NY, 153-160. DOI= http://doi.acm.org/10.1145/1242531.1242555 </nowiki>

[7]<nowiki>Liedtke, J. 1995. On micro-kernel construction. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (Copper Mountain, Colorado, United States, December 03 - 06, 1995). M. B. Jones, Ed. SOSP '95. ACM, New York, NY, 237-250. DOI= http://doi.acm.org/10.1145/224056.224075 </nowiki>

[8]<nowiki>Microkernel verses monolithic kernel
http://www.vmars.tuwien.ac.at/courses/akti12/journal/04ss/article_04ss_Roch.pdf - Roch</nowiki>

I will site it/reference it better later

== Unsorted ==
An overview of exokernels,virtual machines, microkernels *[http://www2.supchurch.org:10999/files/school/classes/CSCI4730/Lectures/grad-structures.ppt Overview](Power Point) 
Should not be used as a source but an overview.

The original paper on [http://portal.acm.org/citation.cfm?id=224076 Exokernels] --[[User:Gautam|Gautam]] 22:39, 6 October 2010 (UTC)

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Outlining some main features here as I see them.

I found that the exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction. They have the same architecture with the basic functionality contained in the kernel to manage everyone. As the exokernel "gives" the resource to the application it can use the resource in isolation of other applications (until forced to shared) much like VMs receive their resources, either partitioned or virtualized, and execute as if its running on its own machine. There is this similar notion of partitioning the resources among applications/OS and allowing them to take control of what they have.

I'll locate some references later on. --[[User:Slay|Slay]] 15:00, 7 October 2010 (UTC)

I'm just going to post my answer for question 1 on the individuel assignment and hope it helps. --[[User:Aellebla|Aellebla]] 15:06, 12 October 2010 (UTC)

The design of the micro kernel was to take everything they could out of the Kernel and put it into a process. For ex, networking would be put into a process instead of staying in the kernel. The micro kernel dev's tried to keep lots of things in user space for efficiency. But one major problem with this is there would be a large amount of moving from a process to the kernel to user space and back again and this is a costly, non efficient process.It was an application specific OS, there was no multiplexing. With a virtual machine you are not virtualizing apps like with a microkernel but virtualizing an entire Operating System. This is very heavy however but the benefits are that it‟s easy and all the standard OS features are there whereas in a microkernel setup they would not all be there and this can be seen as a compromise.

Exokernels can be seen as a compromise to virtual machines and microkernels because virtual machines emulate and exokernels do not. When you emulate something you hide a lot of the actual information because you wouldn‟t be able to see the „real‟ hardware. If we look at a virtual box setup running Linux, and we go look at all the hardware, it will be displayed as fake hardware.

Maybe we can have an introduction - paragraph or so on each type - then similarities - differences - and the compromise. I am going to do some research and writing this weekend and I will put some up -- Jslonosky

btw in my page (i guess you can call it that) i have some resources i have found --[[User:Asoknack|Asoknack]] 15:50, 8 October 2010 (UTC)
- Wow, nice man. I will go ahead and write up the descriptive paragraphs on each kernel and virtual machine if no one minds. --Jslonosky

I think we should divide up the paragraphs and proofread each others instead. (Are there only 4 of us?) I don't have much time to work on this today though but I'll try to work on it tomorrow morning. - Slay

Sure guy. That sounds good. There should be 5 or 6 of us though.. . Oh well. Their loss. I will do some before or after work today. Ill start with Microkernel since there is not a large amount of info here, and so we don't overlap each other - JSlonosky

yeah i think there was more like 7 of us btw if any one has any more information feel free to add it would be nice if you add the references so that way citing is really easy on acm.org it will auto give you the citation info (where it says Display Formats click on ACM Ref and new window with the citation info auto pop's up) --[[User:Asoknack|Asoknack]] 02:28, 11 October 2010 (UTC)

I added an outline of the similarities and differences. Add any more that I missed. These are from observations so I don't have any resources. -Slay
That's probably fine. Our textbook probably outlines some of them, so I am sure we can find a few there - JSlonosky

Talked to the teacher today and for VM he said we should focus on the implementation such as Xen and VMware , he also said to talk about para virtualization --[[User:Asoknack|Asoknack]] 18:42, 12 October 2010 (UTC)

A paper about emulation and paravirtualization [http://portal.acm.org/citation.cfm?id=1189289&coll=GUIDE&dl=GUIDE&CFID=105648137&CFTOKEN=47153176&ret=1#Fulltext link] - Slay

Oh no big words. Sorry about the Microkernels not done yet. Working on an outline now. Finally found how to access the ACM through carleton. Gawd.
I am planning an outline, quick bit about kernels in general, (maybe mention monolith kernels?), and what microkernels do.
I see the microkernel outline info and a reference ( Whomever did that == hero: true) about the scheduling and the Memory management. Should that be included in kernels in general and then mention what microkernels build upon/change? - JSlonosky

Sorry late to the party here. My mistake was not checking the discussion page when I checked in. I don't want to trample anyone's current work but I don't see any work on the final essay done. I would love to help just need to know where I can step in so as to not screw anyone else up. -- [[User:Cling|Cling]]

I don't think I'll be able to write up something for the final essay, even though I suggested splitting it. I'll do research tonight though on the paravirtualization. If I find the time, I'll try to write something. Sorry about that. --[[User:Slay|Slay]] 21:52, 13 October 2010 (UTC)

We all have 3004 to do too, man. I do not think anyone has chosen to do Virtual Machine section yet, or the Exokernel itself. But the contrast paragraph and the intro is chosen, and intro is done. Microkernel and kernel will be done in a hour I hope. -- JSlonosky

I can attempt to write up anything, the issue is I don't have any context on what to write, how do I tie it in to the rest of the essay? I only have a Japanese Quiz tomorrow morning then I should be good to write anything up for the rest of the day. As someone who has already written part of the essay, and assuming I attempt the exokernel section, how much do you think I should write? Should it just be about exokernel or should there be comparisons to the other topics? Thanks --[[User:Cling|Cling]] 23:14, 13 October 2010 (UTC)

Go with the Exokernel itself. Slade is getting off work in a hour and we can double check what he is doing then. We can put it together tomorrow sometime, and fill in the other stuff. - JSLonosky

I'll attempt to work on VM tonight, then. I would feel so bad if I didn't write anything. -Slay

Still wondering how much to write, I think we should decide on a decent word count or length so we don't have one short section (which would probably be mine) and/or one massive section that dwarfs all the others. If anyone has already written a section could you post your word count so we can aim to be around there, it would obviously be just a recommendation but it's just better to be on the safe side and have everything uniform. I haven't seen any formal requirements for the essay but I could be wrong, I also haven't been to class in a while. --[[User:Cling|Cling]] 23:33, 13 October 2010 (UTC)

Yeah Slay, VM probably doesnt have much to write about. Get something down, and we can go over it. CLing, Just write what you think. There is not a lot to go over if I write kernel/microkernel well enough. What is a exokernel? exokernel was an even lower-level design than the microkernel, closer to the hardware without abstraction, basically (As said by Slade). I will probably end up with 500 or a bit more words. -- JSlonosky

Sound off!

Who's actually reading this? Add your name to the list...

Rovic P.
Jon Slonosky
Corey Ling
Steph Lay

== The Essay ==

Let's actually breakdown the essay into components then write it here.

I'd like to go along the premise that microkernels and and virtual machines are "weaker" than exokernels in design for the essay. If anyone has any objections, add it here.

-Slade

what do you mean by "weaker"(i think you mean exokernels' takes the best of both worlds ) --[[User:Asoknack|Asoknack]] 02:45, 13 October 2010 (UTC)

What I mean by weaker is that we should focus on the things microkernels and virtual machines may not do as well compared to a system based off an exokernel design and then focus on how an exokenenel can take the best of both worlds. Please choose which section you will work on, that's not to say it'll be the only part you do, but rather we'll all contribute to each part please. 1 day left.
-Slade

...to the extent that exokernels be seen as a compromise between virtual machines and microkernels.
-I'll work on the initial intro. -Slade

3 paragraphs that prove it
Explain how the key design characteristics of these three system architectures compare with each other.

intro/thesis statement -Rovic P.

In Computer Science, the kernel is the component at the center of the majority of operating systems. The kernel is a bridge for applications to access the hardware level. It is responsible for managing the system's resources such as memory, disk storage, task management and networking. It is how the kernel goes about such management and its connections that we are comparing Exokernels to Microkernels and Virtual Machines. In the Exokernel conceptual model, we can see exokernels become much smaller than microkernels since by design, they are tiny and strive to keep functionality limited to protection and multiplexing of resources. The Virtual Machine Implementation of virtualizing all devices on the system may provide compatibility, but it adds a layer of complexity within the system less efficient than a real machine as it accesses the hardware indirectly. It can be observed how the exokernel provides low level hardware access and provide custom abstraction to those devices to improve program performance as opposed to a VM's implementation. The exokernel concept has a design that can take the better concepts of microkernels and virtual machines to the extend that exokernels can be seen as a compromise between a virtual machine and a microkernel.

Paragraph 1 -Microkernel -Jon S.

The kernel is the most important part of an operating system. Without the kernel, an operating system could not function.

A kernel is the lowest level section of an operating system. It has the most privileges of the system. It runs along side of the ‘user space’. It is in the ‘user space’ where a user has access and where the user can run its applications and libraries.[8] This leaves the kernel with the need to manage the other necessary processes such as the File Systems and the process scheduling. The kernel is layered with the most authoritative process on its lowest level.[8] A monolithic kernel, a kernel that contains all mandatory processes within itself, was the common kernel type of the earlier versions of today’s operating systems utilized. However, this architecture had problems. [8] If the kernel needed to be updated with more code, or a fix for the system, the entire kernel would need to be compiled, and due to the amount of processes within it, it would take an inefficient amount of time. This is where a microkernel becomes practical.

The concept of a microkernel is reduce the code within the kernel, and it is only included in the kernel if it would affect the system in any way, for example for performance and efficiency reasons. [7] So, a microkernel is a kernel that has a reduced amount of mandatory software within itself. This means that it contains less software that it has to manage, and has a reduced size. A microkernel that emerged at the end of the 1980’s to the early 1990’s has the structure that processes like the File Systems and the Drivers are removed from it, leaving the kernel with process control and input/out control, and interrupts. [8] This new structure makes the system much more modular, and easier to provide solutions. If a driver must be patched or upgraded, the kernel does not need to be recompiled. [7] The old driver can be removed, and while the device waits for the system to recognize it, the operating system replaces the driver. This lets real-time updating, and it can be done while the computer is still functional. This can reduce the complete crash of the system. If a device fails, the kernel will not crash itself, like a monolithic kernel would. The microkernel can reload the driver of the device that failed and continue functioning. [7]

Want more on the scheduling? I can do that if wanted. -key note on exokernel's mutiplexing vs microkernel's messaging, exo more efficient so perhaps running with the idea that messaging b/w processes not necessarily the ideal way need to also start outlaying weaknesses in the design as well in order to play up the idea that an exokernel just does it better

Paragraph 2 -Virtual Machine -Steph L.

A Virtual Machine, or VM, is a software abstraction of a physical machine. This entails virtualization of the physical machines resources in order to share them among OS run in the VM. Virtualizing these resources allow the OS to run as if it were on a full machine when, in reality, it is actually running in a virtualized environment on top of a hostOS, the OS actually running on the machine, sharing the resources.

Virtual Machines generally contain two key components, the Virtual Machine Monitor, or VMM and the VM.

The VMM, also known as the hypervisor, manages the virtualization of the physical resources and the interactions with the VM running on top. In other words, it mediates between the virtualized world and the physical world, keeping them separate and monitoring their interactions with each other. The hypervisor is what allows the VM to operate as if it were on its own machine by handling any requests to resources and maintaining these requests with what has actually been provided to the VM by the hostOS. The hostOS is provides management for the VMM as well as physical access to devices, hardware and drivers.

The VM is what contains the OS we are running through virtualization. This OS is called the guestOS and it will only be able to access any resources that have been made available to the VM by the hostOS. Otherwise, the guestOS will not know about any other resources and does not have direct access to physical hardware. This will be taken care of by the VMM.

[ I'll need to do some more research on the types of virtualization though before I can discuss that. If anyone has more information on them to put up in the points that would be helpful but I'll get to it right after class tomorrow morning. ]

Paragraph 3 -Exokernel -Corey L

Paragraph 4 - Contrast/Compromise --[[User:Asoknack|Asoknack]]

Conclusion - Jon S. - Only a sentence per paragraph, excluding Intro

Sweet. Looks like we got it covered. We should read each others parts and put suggestions and edits. One of us should try and change it to one style if there are contradictions. And to put it on the main page. We can figure that out tomorrow. - Jon S

Once the other parts are up and you see anything you know of as a good reference to back it up, put the link so we can use it. -Slade

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T03:49:13Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T03:08:28Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T00:30:50Z

Slade: /* Unsorted */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-14T00:01:29Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-13T23:58:56Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-13T00:48:56Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-11T15:43:32Z

Slade: /* The Essay */

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-11T15:38:19Z

Slade: /* The Essay */ new section

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-07T13:02:48Z

Slade:

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to direct access through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

Talk:COMP 3000 Essay 1 2010 Question 1

2010-10-07T12:50:38Z

Slade: Created page with 'Exokernel- Minimalistic abstractions for developers Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give th…'

Exokernel-
Minimalistic abstractions for developers
Exokernels can be seen as a good compromise between virtual machines and microkernels in the sense that exokernels can give that low level access to developers similar to microkernels through a protected layer and at the same time can contain enough hardware abstraction to allow similar benefit of hiding the hardware resources to application programs.
Exokernel – fewest hardware abstractions to developer
Microkernel - is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system
Virtual machine is a simulation of any or devices requested by an application program
Exokenel – I’ve got a sound card
Virtual Machine – I’ve got the sound card you’re looking for, perfect virtual match
Microkernel – I’ve got sound card that plays Khazikstan sound format only
MicroKernel - Very small, very predictable, good for schedualing (QNX is a microkernel - POSIX compatable, benefits of running linux software like modern browsers)

This is some ideas I've got on this question, please contribute below
-Rovic

COMP 1501 Presentations

2008-12-03T19:37:18Z

Slade: /* 14:30-15:00, Wednesday, December 3, 2008 */

Please add your names to a presentation time slot below. To sign up, click "edit" beside the time you want and add your names. (Please do not edit this entire page please, that is a recipe for editing conflicts.)

All presentations are in 5151 HP (the Gaming lab).

==Monday, December 1, 2008==

===10:00-10:30, Monday, December 1, 2008===

Barbeau, Oda: Murray Christopherson, Daniel Kelly - Raven's Trinity

===10:30-11:00, Monday, December 1, 2008===

Barbeau, Oda: Henry Irving, Dan Peeler

===11:00-11:30, Monday, December 1, 2008===

Barbeau, Oda:
Robert Wolfe and Paul Mauviel

===11:30-12:00, Monday, December 1, 2008===

Barbeau, Oda: Drew Martin, Eva Demers-Brett

===13:00-13:30, Monday, December 1, 2008===

Barbeau, Gail: Nicolas Porter and Idris Cameron [LawnMaster]

===13:30-14:00, Monday, December 1, 2008===

Barbeau, Gail: Daniel Hockey and Anthony D'Angelo

===14:00-14:30, Monday, December 1, 2008===

Barbeau, Gail: Neil Prowse and David Xiong

===14:30-15:00, Monday, December 1, 2008===

Barbeau, Gail: JP Landry, Derek Langlois

==Tuesday, December 2, 2008==

===10:00-10:30, Tuesday, December 2, 2008===

Somayaji, Doherty: Jeff Francom, Spencer Elliott

===10:30-11:00, Tuesday, December 2, 2008===

Somayaji, Doherty: Spencer Winson & Sebastian Podlesny

===11:00-11:30, Tuesday, December 2, 2008===

Somayaji, Gail: Nick Tierney and Jake Hendrick

===11:30-12:00, Tuesday, December 2, 2008===

Somayaji, Doherty: Erik Bitmanis

===13:00-13:30, Tuesday, December 2, 2008===

Barbeau: Michael Evans, Robert Theoret

Somayaji, Doherty: Andrew Erdeg and Sebastian Schneider - Blood Hospital

===13:30-14:00, Tuesday, December 2, 2008===

Barbeau, Gail: Jonathon Slonosky, Shawn Reichert

Somayaji, Doherty: Adam Saunders and Brendan Cooper (Bob The Tesseract)

===14:00-14:30, Tuesday, December 2, 2008===

Barbeau, Gail: Tom Goldsmith and Rob Lavoie - The Fall of the Risen

Somayaji, Oda: Trevor Malone and Wesley Lawrence - Rise of the Fallen

===14:30-15:00, Tuesday, December 2, 2008===

Barbeau, Gail: Bruno Colantonio, Kevin McCreedy - Tuk's Revenge

Somayaji, Oda: Daniel Sont, Mattieu Colverson

===15:00-15:30, Tuesday, December 2, 2008===

Somayaji, Gail: Jeffery Luangpakham and Armando Taucer

===15:30-16:00, Tuesday, December 2, 2008===

Somayaji, Gail: Matthew Chou and Yingfan Peng

==Wednesday, December 3, 2008==

===13:30-14:00, Wednesday, December 3, 2008===

Barbeau, Oda: sami zabarah ,yuanbin tang and song chang

Somayaji, Doherty: Julie Powers and Austin Chamney

===14:00-14:30, Wednesday, December 3, 2008===

Barbeau, Oda:Mira Richard-Fioramore et Joel Theresine

Somayaji, Doherty: Scott Bennett and Nick Shires

===14:30-15:00, Wednesday, December 3, 2008===

Barbeau, Oda: David Krutsko and Stephany Lay

Somayaji, Doherty: Rovic Perdon, Rodrick McDonald, Clayton Shier

===15:00-15:30, Wednesday, December 3, 2008===

Somayaji, Doherty: Daniel Beimers and Arthur Marshall

===15:30-16:00, Wednesday, December 3, 2008===

Somayaji, Doherty: Martin Kugler & Paul MacDonald