Soma-notes - User contributions [en]

Locus, V, Mach

2008-02-02T18:25:39Z

Rgould: Add notes for Yuri

Locus, V, Mach

2008-01-16T20:54:46Z

Rgould:

==Readings==

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System."]

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System."]

These two papers describe V and LOCUS, two early distributed operating systems.

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/young-mach-dual.pdf Michael Young et al., "The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System."]

This paper describes some key ideas behind Mach, a seminal microkernel-based operating system. The design of Mach informs later work in distributed operating systems. For more background on Mach, I suggest reading [http://en.wikipedia.org/wiki/Mach_kernel Wikipedia's article on Mach].

==Debate==
Debate about the pros and cons of RPC. Topic: RPCs are the right foundation for the distributed applications (on today's Internet).

===Pros===
# Easy to use
#* makes remove procedure calls look like local calls
#* local is easy
#* therefore remove is easy with RPC - good!
# (Counter to first con) Use standards, problem goes away
# Easy to debug
# Avoids complexity of protocol design
# RPC can be abstracted from the protocol design
# (Counter to fourth con) Use threads for interactivity. Manage complexity by doing things pairwise.
# (Counter to fifth con) - wrong!
# (Counter to seventh con) - Use standards (ORBs in CORBA, etc.) for authentication
# (Counter to eighth con) - But you specify the API
# (Counter to ninth con) - It's the same with other methods. But RPC makes it easier to do.

===Cons===
# Language/implementation specific
# (Counter to third pro) Not easy to debug. example: NullPointerException thrown on server, containing line of erroneous code running on the server.
# Counter to fifth pro) But there is more overhead - slower
# Synchronous - wrong model for large, distributed applications, and also bad for users
# Limited scalability - hard to do well
# Limited control of communication details
# Authentication is opaque
# Larger attack surface area
# Poor match to listening (hard for server to communicate information back to client)
# Backwards compatibility is difficult
# RPC is not a "natural" interaction style - message passing is more similar to human communication
# Too much to set up for simple applications
# Less flexible

===Comments===
RPC makes the simple things easy, but the most difficult aspects are still prevalent. Good to use if you are only using the common case. For example, a cluster that is isolated from the Internet.

==Questions to be discussed==

# What did the (technical) world look like when this distributed OS was designed & implemented?
# What are the key design features of the OS? What are the key compromises?
# To what extent is their system a "distributed operating system"?
# What is the basic argument for their design choices? What evidence do they cite in their favor?
# To what extent do you "believe" their design choices were right? Why?
# To what extent does their design work in the context of the modern Internet? Discuss wins and limitations.
# What concepts from the paper have been adopted by today's systems? What concepts should be adopted? What should be ignored?

Locus, V, Mach

2008-01-16T20:48:49Z

Rgould:

==Readings==

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System."]

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System."]

These two papers describe V and LOCUS, two early distributed operating systems.

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/young-mach-dual.pdf Michael Young et al., "The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System."]

This paper describes some key ideas behind Mach, a seminal microkernel-based operating system. The design of Mach informs later work in distributed operating systems. For more background on Mach, I suggest reading [http://en.wikipedia.org/wiki/Mach_kernel Wikipedia's article on Mach].

==Debate==
Debate about the pros and cons of RPC. Topic: RPCs are the right foundation for the distributed applications (on today's Internet).

===Pros===
# Easy to use
#* makes remove procedure calls look like local calls
#* local is easy
#* therefore remove is easy with RPC - good!
# (Counter to first con) Use standards, problem goes away
# Easy to debug
# Avoids complexity of protocol design
# RPC can be abstracted from the protocol design
# (Counter to fourth con) Use threads for interactivity. Manage complexity by doing things pairwise.
# (Counter to fifth con) - wrong!
# (Counter to seventh con) - Use standards (ORBs in CORBA, etc.) for authentication
# (Counter to eighth con) - But you specify the API
# (Counter to ninth con) - It's the same with other methods. But RPC makes it easier to do.

===Cons===
# Language/implementation specific
# (Counter to third pro) Not easy to debug. example: NullPointerException thrown on server, containing line of erroneous code running on the server.
# Counter to fifth pro) But there is more overhead - slower
# Synchronous - wrong model for large, distributed applications, and also bad for users
# Limited scalability - hard to do well
# Limited control of communication details
# Authentication is opaque
# Larger attack surface area
# Poor match to listening (hard for server to communicate information back to client)
# Backwards compatibility is difficult
# RPC is not a "natural" interaction style - message passing is more similar to human communication
# Too much to set up for simple applications
# Less flexible

==Questions to be discussed==

# What did the (technical) world look like when this distributed OS was designed & implemented?
# What are the key design features of the OS? What are the key compromises?
# To what extent is their system a "distributed operating system"?
# What is the basic argument for their design choices? What evidence do they cite in their favor?
# To what extent do you "believe" their design choices were right? Why?
# To what extent does their design work in the context of the modern Internet? Discuss wins and limitations.
# What concepts from the paper have been adopted by today's systems? What concepts should be adopted? What should be ignored?

Locus, V, Mach

2008-01-16T20:37:24Z

Rgould:

==Readings==

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System."]

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System."]

These two papers describe V and LOCUS, two early distributed operating systems.

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/young-mach-dual.pdf Michael Young et al., "The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System."]

This paper describes some key ideas behind Mach, a seminal microkernel-based operating system. The design of Mach informs later work in distributed operating systems. For more background on Mach, I suggest reading [http://en.wikipedia.org/wiki/Mach_kernel Wikipedia's article on Mach].

==Debate==
Debate about the pros and cons of RPC. Topic: RPCs are the right foundation for the distributed applications (on today's Internet).

===Pros===
# Easy to use
#* makes remove procedure calls look like local calls
#* local is easy
#* therefore remove is easy with RPC - good!
# (Counter to first con) Use standards, problem goes away
# Easy to debug
# Avoids complexity of protocol design
# RPC can be abstracted from the protocol design
# (Counter to fourth con) Use threads for interactivity. Manage complexity by doing things pairwise.
#

===Cons===
# Language/implementation specific
# (Counter to third pro) Not easy to debug. example: NullPointerException thrown on server, containing line of erroneous code running on the server.
# Counter to fifth pro) But there is more overhead - slower
# Synchronous - wrong model for large, distributed applications, and also bad for users
# Limited scalability - hard to do well
# Limited control of communication details
# Authentication is opaque
# Larger attack surface area

==Questions to be discussed==

# What did the (technical) world look like when this distributed OS was designed & implemented?
# What are the key design features of the OS? What are the key compromises?
# To what extent is their system a "distributed operating system"?
# What is the basic argument for their design choices? What evidence do they cite in their favor?
# To what extent do you "believe" their design choices were right? Why?
# To what extent does their design work in the context of the modern Internet? Discuss wins and limitations.
# What concepts from the paper have been adopted by today's systems? What concepts should be adopted? What should be ignored?

Locus, V, Mach

2008-01-16T20:23:52Z

Rgould:

==Readings==

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System."]

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System."]

These two papers describe V and LOCUS, two early distributed operating systems.

[http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/young-mach-dual.pdf Michael Young et al., "The Duality of Memory and Communication in the Implementation of a Multiprocessor Operating System."]

This paper describes some key ideas behind Mach, a seminal microkernel-based operating system. The design of Mach informs later work in distributed operating systems. For more background on Mach, I suggest reading [http://en.wikipedia.org/wiki/Mach_kernel Wikipedia's article on Mach].

==Debate==
Debate about the pros and cons of RPC. Topic: RPCs are the right foundation for the distributed applications (on today's Internet).

==Pro==
# Easy to use
#* makes remove procedure calls look like local calls
#* local is easy
#* therefore remove is easy with RPC - good!
# (Counter to first con) Use standards, problem goes away
# Easy to debug
# Avoids complexity of protocol design
# RPC can be abstracted from the protocol design
# (Counter to fourth con) Use threads for interactivity. Manage complexity by doing things pairwise.

==Con==
# Language/implementation specific
# (Counter to third pro) Not easy to debug. example: NullPointerException thrown on server, containing line of erroneous code running on the server.
# Counter to fifth pro) But there is more overhead - slower
# Synchronous - wrong model for large, distributed applications, and also bad for users

==Questions to be discussed==

# What did the (technical) world look like when this distributed OS was designed & implemented?
# What are the key design features of the OS? What are the key compromises?
# To what extent is their system a "distributed operating system"?
# What is the basic argument for their design choices? What evidence do they cite in their favor?
# To what extent do you "believe" their design choices were right? Why?
# To what extent does their design work in the context of the modern Internet? Discuss wins and limitations.
# What concepts from the paper have been adopted by today's systems? What concepts should be adopted? What should be ignored?

Proposal Meeting Schedule

2007-10-26T03:33:33Z

Rgould:

==Tuesday, Oct. 30th==

{| class="wikitable"
| 10:00 AM
|
|-
| 10:10 AM
|
|-
| 10:20 AM
|
|-
| 10:30 AM
|
|-
| 10:40 AM
|
|-
| 10:50 AM
|
|-
| 2:30 PM
|
|-
| 2:40 PM
|
|-
| 2:50 PM
|
|-
| 3:00 PM
|
|-
| 3:10 PM
|
|-
| 3:20 PM
|
|}

==Wednesday, Oct. 31st==

{| class="wikitable"
| 10:30 AM
|
|-
| 10:40 AM
|
|-
| 10:50 AM
|
|-
| 11:00 AM
|
|-
| 11:10 AM
|
|-
| 11:20 AM
|
|-
| 11:30 AM
|
|-
| 11:40 AM
|
|-
| 11:50 AM
|
|-
| 12:00 PM
| Neil Dickson
|-
| 12:10 PM
|
|-
| 12:20 PM
| Richard Gould
|}

==Thursday, Nov. 1st==

{| class="wikitable"
| 1:30 PM
|
|-
| 1:40 PM
|
|-
| 1:50 PM
|
|-
| 2:00 PM
|
|-
| 2:10 PM
|
|-
| 2:20 PM
|
|}

==Tuesday, Nov. 6th==

{| class="wikitable"
| 10:00 AM
|
|-
| 10:10 AM
|
|-
| 10:20 AM
|
|-
| 10:30 AM
| David Tremayne
|-
| 10:40 AM
|
|-
| 10:50 AM
|
|-
| 2:30 PM
| Maria Krol
|-
| 2:40 PM
| Adam Becevello
|-
| 2:50 PM
| Jeff Snell
|-
| 3:00 PM
|
|-
| 3:10 PM
|
|-
| 3:20 PM
|
|}

==Wednesday, Nov. 7th==

{| class="wikitable"
| 10:30 AM
| Kenneth Chan
|-
| 10:40 AM
|
|-
| 10:50 AM
|
|-
| 11:00 AM
|
|-
| 11:10 AM
| Adam McNamara
|-
| 11:20 AM
|
|-
| 11:30 AM
|
|-
| 11:40 AM
| Geoffrey Johnson
|-
| 11:50 AM
|
|-
| 12:00 PM
|
|-
| 12:10 PM
|
|-
| 12:20 PM
|
|}

==Thursday, Nov. 8th==

{| class="wikitable"
| 1:30 PM
|
|-
| 1:40 PM
|
|-
| 1:50 PM
|
|-
| 2:00 PM
|
|-
| 2:10 PM
|
|-
| 2:20 PM
|
|}

Operating System Organization

2007-09-24T03:25:22Z

Rgould:

==Operating System Organization==

===Device Management===

How do kernels communicate with devices such as a network card? How do drivers for such devices fit into the kernel? We need a mechanism to allow applications to communicate with the devices. Most kernels use a form of message passing, often using a registration system. For example, a network card device driver would register itself with the kernel and identify that it is in fact a network card (as opposed to say, a mouse). MS DOS used interrupt handlers instead.

Glenn's talk focused mostly on the how the Linux Kernel works.

====File Abstraction in Device Management====
In Linux there exists a "/proc" directory which contains special information. This directory does not actually exist on disk. When the kernel receives a request to read a file in one of these directories, it retrieves system information and serves it up as a file. For example, executing more /proc/cupinfo gives:

<pre>
$ more /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 13
model name : Intel(R) Pentium(R) M processor 1.73GHz
stepping : 8
cpu MHz : 798.000
cache size : 2048 KB
...
</pre>

Each process that is currently running on the system gets its own directory in /proc, with the process ID (pid) as the directory name. For example for process with pid 2, there exists "/proc/2/" which contains more information about that process.

=====/dev=====

The "/dev" directory actually exists on the file system and contains entries for devices (called nodes). For example, the first hard drive on the system might reside in "/dev/hda/". Each device entry has a major node number and a minor node number. For example, the hard drive specified by "/dev/hda" might have major node number "3" and minor node number "0". At first the node numbers were pre-defined and there could be no more than 255 of them. These major/minor node numbers are used to link the specific device types into the kernel. These nodes existed in "/dev" even if the devices were not connected to the system.

=====devfs=====

This was eventually replaced with a new system called "devfs" (device file system), which was a pseudo-file system similar to /proc. Devfs is implemented in the kernel and knows about the currently available hardware. Some problems still existed with this system: it was implemented in the kernel, and thus a change to hardware required an update to the kernel; and the major and minor node numbers were still fixed in the kernel. It would be nice to dynamically reassign major nodes to unknown devices that are actually present on the system. Devfs also prevented the renaming of nodes in /dev. For example, previously one could rename /dev/hda to /dev/cdrom, but it would still actually point at hard-drive a. This behaviour was prevented in devfs.

=====udev=====

Another problem existed with hot-pluggable devices (such as USB devices). Minor node numbers were assigned by the kernel in the order by which they were discovered. Devices might have different node numbers after a reboot. Also no notifications occur when a device is connected or disconnected from the system.

Devfs has since been replaced by a new system named "udev", which was implemented in the user space, not the kernel space. The ability to rename nodes in /dev was enabled again by udev. The issue regarding hot-pluggable devices was addressed by permitting minor node numbers to be dynamically assigned. udev also notifies applications when hardware is connected or disconnected. Network cards are a special case - the kernel knows about network protocols, so network cards must be accessed using a different interface.

=====Other files, pipes and sockets=====

An example call for opening the CD-ROM may look something like:
<pre>
handle = open(/dev/cdrom, ...)
</pre>
This is call to the kernel which will use the cd-rom devices drivers to read from the disc.

Pipes and sockets both operate as files and support the basic operations
* open
* read
* write
* close

Pipes are used for inter-process communication. Each process can open one end of the pipe, and then they can read or write to it.

Sockets are used in a similar manner to communicate over a network.

===Kernel Development===

Standard development tools aren't always helpful when debugging during kernel development. How can you debug a kernel that crashes before the display drivers work? The Linux kernel outputted Morse code to the LED lights on the keyboard.

Often developers must work around bugs in hardware, as it is usually much cheaper to fix it in software than to change the hardware design.

===Process and Thread Management===

Context switching between different process is very expensive in terms of execution time. Different situations call for different strategies for managing context switches. For example, consider terminals and servers. Server systems can get away with fewer context switches, as they can completely serve up a request for a web-page before moving on to the next request. On a terminal, the user is present and expects instant feedback. If the mouse is moved, and the process that moves the mouse pointer does not respond quickly, the user will perceive that the machine is slow, or unresponsive.

===Memory Management===

Chapter 3 gives just a brief introduction to memory management.

Processes have their own virtual memory map. P3s and P4s used 32-bit addressing, which gave a maximum address space of 4GB. Note that there may not even be 4GB of physical RAM available to be used. When a process requests memory that is not currently in RAM, the operating system must retrieve it from disk (aka paging).

The operating system also needs to protect the kernel's memory space from other applications. Supervisor (root) vs user mode determines the level of access to memory.

===Kernel Design===

Monolithic vs micro - how much stuff should be implemented in the kernel. Microkernel design means that processes and applications do more work, which requires more context switching, but this permits the kernel to be more reliable.

An example monolithic kernel might include things such as:
* network driver
* display
* clock

A micro kernel might only include the minimal items:
* memory allocation
* process switching

===File Systems===

Consist of many items:
* directories
* files
* device nodes
* links (in Windows these are called shortcuts)
* pipes

In DOS, only directories, files and device nodes were used. In Windows, they are well hidden under the path "\\" or they are given special names, such as "AUX", "COM0" or "LPT0".

Operating System Organization

2007-09-24T03:11:31Z

Rgould:

Operating System Organization

2007-09-24T02:55:12Z

Rgould: