Soma-notes - User contributions [en]

Operating Systems 2018F Lecture 19

2018-11-23T08:50:11Z

R.samanfar: /* Notes */

==Video==

Video for the lecture given on November 16, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec19-20181116.m4a is now available].

==Notes==

Linux source code links:

* [https://elixir.bootlin.com/linux/v4.19.2/source/fs/read_write.c#L107 use of spin_lock() and spin_unlock()]
* [https://elixir.bootlin.com/linux/v4.19.2/source/include/linux/fs.h#L895 spinlock declaration]
* [https://elixir.bootlin.com/linux/v4.19.2/source/include/linux/spinlock.h#L321 spinlock init]

== Question re Pages for Assignment ==

*Pages are virtual
*Put into a frame which is physical memory
*when allocating memory: allocating frames & putting pages into these frames
*Page is a chunk of process memory

==Remember Module from Tutorial 7 ==

#What happens when more than one process is trying to write to dev/remember at same time?
#*Get race condition
#*Race condition located in <code>remember_write()</code>
#*Condition is basically the whole function
#*Conflict in <code>free_saved_data()</code>, <code>init_saved_date()</code>, and when writing to it.
#**Examples: freeing twice, first time will work but will try to free pages again, but nothing to free

#How can we fix in order to allow for concurrency?
#*One solution: add locking mechanism, mutual exclusion to access data structures (lines 35-39 need to be locked)
#*Using sleep not a good idea

#Simplest Solution: Prevent Crashing Kernel
#*Real problem for kernel crashing is freeing and allocating memory
#*Canâ€™t use mmap, which is user-space command, in the kernel
#*Why does <code>remember</code> allocate data on every write?
#**Every time freeing memory, then initializing new, but its the same size
#**Why throw away? Just erase it, or replace with zeros
#*So, move allocation into <code>remember_init()</code>
#*Move deallocation into the <code>remember_exit()</code> (when kernel module unloaded)
#*Can still have problems, but avoids crashing!
#*From here we can then add locking mechanisms

#Another Solution: Exclusivity
#*Make it exclusive
#*Only have concurrency problems with shared data
#*Make data exclusive to a process, or a user, etc.
#*If can avoid locks, should, because they can cause problems
#*Locks are error prone

#Locks in Kernel Space
#*We have to rely on kernel based mechanism, can't rely on man page
#*There is Documentation online for kernel locking mechanism, but be careful, some info outdated
#*Multiple kinds of locking, canâ€™t just search lock in source code for "lock" because accessing primitive info
#*Need an example to follow & base code on
#* ''INODE'' uses <code>spinlock_t</code> & <code>rw_semaphore</code>

## <code>spinlock.h</code>
#*What does it do?
#*Similar to loop structure from '''Test 1'''
#*Creates a do-while loop to lock
#*A bunch of implementations, <code>spinlock_up.h</code>

## Question: How do normal files handle locking?
#*Checks in <code>Fs.h<code> for struct inode
#*Uses <code>spin_lock</code>

#Concurrent Remember Module
#*First: move allocation & deallocation to <code>remember_init()</code> & <code>remember_exit()</code>
#*Add: <code>static spinlock_t saved_data_lock</code> to initialized structures (line 41)
#*In <code>init_saved_data()</code>, add initialization of spinlock
#** <code>spin_lock_init(&saved_data_lock);</code> (line 94)
#*In <code>remember_write()</code>, add lock & unlock
#**<code>spin_lock(&saved_data_lock)</code> (line 131)
#**<code>spin_unlock(&saved_data_lock)</code> (line 134)

##Do we need lock around read as well?
#*Yes, otherwise inconsistent data if someone reading while writing
#*In <code>remember_read()</code>
#**<code>spin_lock(&saved_data_lock)</code> (line 74)
#**<code>spin_unlock(&saved_data_lock)</code> (line 77)

##Now build module, but not load directly on device
#*Will it compile? It did!
#*Now test in SSH/VM
#**Uses rsync to copy file to vm, <code>make</code> & then loads modules into kernel
#*Test 1: <code>echo â€œhello thereâ€ > /dev/remember </code>
#**<code>cat dev/remember</code> Works properly, & prints out ''hello there''
#*But we need to test concurrency
#*How?
#**Make child-parent program that just keeps writing with whatever is writen to the command line
#**From command line just keep adding & check with what have
#**Not a definitive test, but passible

== Remember Write Test ==
#Version 1
#*Opens file, writes first argument from command line to file, seeks back to beginning, then loops
#*Need <code>seek</code> to reset offset back to beginning, otherwise would keep writing same thing instead of writing over
#* Works, but printing result and new command line on same line

#Version 2
#*Adds a new line char to end of <code>dev/remember</code>
#*Works

#Final Version
#*Counter <code>i</code> (line 21)
#**Used because otherwise printing out the same thing every time, no way of knowing if actually working properly

#ERROR: Filled up log file
#*Clean out journal log
#**<code>sudo journalctl --flush --rotate</code>
#**<code>sudo journalctl --vacuum-size=1M</code>

#Remember-WriteTest on Remember Module
#*Run 3 processes in background
#* <code>./remember-writetest p1 &</code>
#**works
#* <code>./remember-writetest p2 &</code>
#**get segmentation fault
#* <code>./remember-writetest p3 &</code>
#**process 1 killed
#*<code>cat /dev/remember<code>
#** works
#*So kernel not crashing, but still getting errors
#*Locking successful

==Code==

'''[https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec19/remember-concurrent.zip Download remember-concurrent.zip]'''

===remember.c (concurrent version)===

<source lang=c line>
/*
remember.c

remember module COMP 3000, Carleton University
remembers what is written to it, returns in when read along with
info about how it is stored

License: GPLv2 or later
Author: Anil Somayaji
November 3, 2018

device driver and module code derived from:
https://appusajeev.wordpress.com/2011/06/18/writing-a-linux-character-device-driver/
and
http://pete.akeo.ie/2011/08/writing-linux-device-driver-for-kernels.html
*/

#include <linux/module.h>
#include <linux/string.h>
#include <linux/fs.h>
#include <linux/device.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/uaccess.h>
#include <linux/mm.h>

#define DEVICE_NAME "remember"
#define CLASS_NAME "comp3000"

static struct class* remember_class = NULL;
static struct device* remember_device = NULL;
static int remember_major;

static struct page *saved_data_page = NULL;
static char *saved_data = NULL;
static unsigned long saved_data_len = 0;
static int saved_data_max = PAGE_SIZE;
static int saved_data_order = 0;

static spinlock_t saved_data_lock;

static int remember_open(struct inode *the_inode, struct file *f)
{
pr_info("Remember: device opened\n");
return 0;
}

static ssize_t remember_read(struct file *f, char *buf, size_t len, loff_t *offset)
{
unsigned long n;
char *error_msg = "Buffer too small.";

pr_info("Remember: read started\n");

if (*offset > 0) {
pr_info("Remember: read non-zero offset, aborting\n");
return 0;
}

if (len < saved_data_len) {
pr_info("Remember: read short buffer\n");
n = strlen(error_msg) + 1; // Include terminating null byte
if (n > len) {
n = len;
}
copy_to_user(buf, error_msg, n);

return n;
} else {
pr_info("Remember: read returning data, %ld bytes\n",
saved_data_len);

spin_lock(&saved_data_lock);
copy_to_user(buf, saved_data, saved_data_len);
*offset = saved_data_len;
spin_unlock(&saved_data_lock);

return saved_data_len;
}
}

void init_saved_data(void)
{
pr_info("Remember: allocating data page");

if (saved_data) {
pr_err("Remember: saved_data already initialized!");
} else {
saved_data_page = alloc_pages(GFP_KERNEL,
saved_data_order);
saved_data = (char *) page_address(saved_data_page);
saved_data_len = 0;
spin_lock_init(&saved_data_lock);

pr_info("saved_data at kernel virtual address %lx",
(unsigned long) saved_data);
pr_info("saved_data_page page struct at address %lx",
(unsigned long) saved_data_page);
}
}

void free_saved_data(void)
{
if (saved_data_page) {
pr_info("Remember: freeing old data page");
__free_pages(saved_data_page, saved_data_order);
saved_data_page = NULL;
saved_data = NULL;
saved_data_len = 0;
}
}

static ssize_t remember_write(struct file *f, const char *buf, size_t len,
loff_t *offset)
{
unsigned long result;

if (*offset > 0) {
pr_info("Remember: write nonzero offset, aborting");

return 0;
}

if (len > saved_data_max) {
len = saved_data_max;
}

pr_info("Remember: write saving data, %ld bytes", len);

spin_lock(&saved_data_lock);
result = copy_from_user(saved_data, buf, len);
saved_data_len = len;
spin_unlock(&saved_data_lock);

*offset = len;

return len;
}

static int remember_release(struct inode *the_inode, struct file *f)
{
pr_info("Remember: device closed\n");
return 0;
}

static struct file_operations remember_fops = {
.open = remember_open,
.read = remember_read,
.write = remember_write,
.release = remember_release,
};

static char *remember_devnode(struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0666;
return NULL;
}

static int __init remember_init(void)
{
int retval;

remember_major = register_chrdev(0, DEVICE_NAME, &remember_fops);
if (remember_major < 0) {
pr_err("failed to register device: error %d\n", remember_major);
retval = remember_major;
goto failed_chrdevreg;
}

remember_class = class_create(THIS_MODULE, CLASS_NAME);
if (IS_ERR(remember_class)) {
pr_err("Remember: failed to register device class '%s'\n",
CLASS_NAME);
retval = PTR_ERR(remember_class);
goto failed_classreg;
}

remember_class->devnode = remember_devnode;

remember_device = device_create(remember_class, NULL,
MKDEV(remember_major, 0),
NULL, DEVICE_NAME);

if (IS_ERR(remember_device)) {
pr_err("Remember: failed to create device '%s'\n", DEVICE_NAME);
retval = PTR_ERR(remember_device);
goto failed_devreg;
}

init_saved_data();

pr_info("Remember: device registered using major %d.\n",
remember_major);

return 0;

failed_devreg:
class_unregister(remember_class);
class_destroy(remember_class);
failed_classreg:
unregister_chrdev(remember_major, DEVICE_NAME);
failed_chrdevreg:
return -1;
}

static void __exit remember_exit(void)
{
free_saved_data();

device_destroy(remember_class, MKDEV(remember_major, 0));
class_unregister(remember_class);
class_destroy(remember_class);
unregister_chrdev(remember_major, "remember");
pr_info("Unloading Remember module.\n");
return;
}

module_init(remember_init);
module_exit(remember_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Anil Somayaji <soma@scs.carleton.ca>");
MODULE_DESCRIPTION("A remember character device module");
</source>

===remember-writetest.c===

<source lang=c line>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>

#define BUFSIZE 8192

int main(int argc, char *argv[])
{
int fd, result, count;
int i = 0;
char buf[BUFSIZE];

while(1) {
count = snprintf(buf, BUFSIZE, "%s: %d\n", argv[1], i);
fd = open("/dev/remember", O_WRONLY);
result = write(fd, buf, count);
close(fd);
i++;
}

/* will never run */
return 0;
}
</source>

Operating Systems 2018F Lecture 18

2018-11-23T08:48:03Z

R.samanfar: /* Notes */

==Video==

The video from the lecture given on November 14, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec18-20181114.m4a is now available].

==Notes==

What is dev/fuse
How did i do a mount

Mount is a priveleged operation normally, it lets you shadow entire portions of the filesystem

fusermount breaks under strace

bin/fusermount is red because of the 's' permission bit.

'''What is the 's' permission bit?'''
s stands for setuid in the owner position, and setgid in the group position.
It allows a program to run with privileges different from the process that did the execve

Normally execve of a process will run as the user (the uid that is currently running)

'''setuid''' gives it the privileges of root instead

'''getuid()''' is who ran the process
'''geteuid()''' is with what privelges is this process running
normally these are equal, but it changes when you run '''setuid'''
(same thing for gid)

If you run memoryll as root, the mnt point and all the files under it are exclusive to root
One of the features of userspace mounting of filesystems, is that they are exclusive to the user that mounted them.

'''getattr()'''
'''system calls'''
writev and read, its a way of using multiple buffers
mount - mounts dev/fuse and fd 3
===In Class===
<pre>
Lecture 18
----------

To Do
-----
strace memoryll
uid vs euid
scheduling and __schedule

Optional reading:
https://utcc.utoronto.ca/~cks/space/blog/unix/RawTtyInputThenAndNow

Scheduling
----------

System calls => enter supervisor mode (kernel) from user mode (processes)

scheduler => enter user mode (processes) from supervisor mode (kernel)

Scheduling algorithms

Challenges of scheduling
- maximize use of resources (throughput)
- minimize wait time (latency)
- minimal computational complexity
scheduling is pure overhead
- make sure "important" things get done
- obey user preferences
- do important system things

Classic scheduling
Batch scheduling
- shortest job first
- lowers latency for shorter jobs but isn't "fair"
- first in, first out
- fair, but can have bad consequences

"time sharing" - preemptible scheduling
- each process gets a "time slice"
- when process is interupted, what to do next?
key question
- is it waiting for I/O?
</pre>

Operating Systems 2018F Lecture 17

2018-11-23T08:47:36Z

R.samanfar: /* Notes */

==Video==

The video from the lecture on November 9, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec17-20181109.m4a is now available].

==Notes==

alloc pages -> frames -> page

''''FUSE'''' - filesystem in userspace
interface for userspace programs to export a filesystem to the LINUX kernel

'''What is a filesystem from a kernel?'''
* A device file (character device, block device), depends on file operations.
* A filesystem is just this scaled up, it should not only include these file operations, but it should also include operations for directories, and mounting, unmounting.
If you can define those, you can make a filesystem

FUSE makes a "special" filesystem that allows you to create other filesystems

Filesystems are abstractions of file operations
Filesystems talk to rest of the kernel through a virtual filesystem interface

'''Are all userspace filesystem in memory?'''
Standard ntfs implementation on most linux distributions, is written in userspace

'''Microkernals vs monolithic kernal'''
microkernel: everything turns into a process
monolithic kernel: it's faster

'''memoryll.py'''
mounting dev/fuse
any file operation is going to this python program

'''remember.c'''
To increase storage, update:
saved_data_max
saved_data_order

Debugging kernel code is difficult, but kernel can be watched while running
===In Class===

<pre>
Lecture 17
----------

Filesystems

What is it?

Well, what's a device file?
- struct file_operations which gives functions for file operations
(read, write, etc)

filesystems just take this idea further

When you access a file
- process does a file-related system call
- kernel calls generic system call function for requested operation
- kernel calls filesystem-specific function for requested operation

e.g.
- process does a write system call on file on an ext4 filesystem
- kernel runs generic write syscall code
- kernel runs ext4 write code

filesystems are abstractions of file and directory-related operations
- allow many instantiations that we call filesystems

filesystems talk to rest of the kernel through a virtual filesystem interface
(VFS)

Sidebar:
Microkernels vs monolithic kernels

Why microkernels
- better security: if a service crashes, it is just a process
- easier development: servers are just a process

Why monolithic kernels
- faster
- you depend on it anyway, so security benefits are illusory
</pre>

Operating Systems 2018F Lecture 12

2018-11-21T20:41:03Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on October 17, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec12-20181017.m4a is now available].

==Notes==
Today: Kernel modules

=== First, Anil's ssh setup: ===
<code>student@anil-vm:</code> // openstack
* why is it named anil-vm?
** edit /etc/hosts and /etc/hostname
<code>anilclass@sutherland:</code> // local machine
* edit /etc/hosts
** add the IP of your VM and give it name
** distributed DNS does this for URLs
* login without password
** ssh key pair, private and public
** files in ~/.ssh/

Public keys?
* sharing won't compromise security

Running X-windows programs on vm:
* <code>ssh -X student@anil-vm</code> // works natively in linux
* on macOS, you can run X-server separately
* on Windows, use x2go

== Whats a kernel module? ==
* kernel code, that you load, into the kernel
* same privileges as other kernel code
* dangerous! load kernel module = anything can happen
** only superuser (root) can load
* NOT a system call

Kernel Oops:
* kernel catches the bad memory access
Kernel crash:
* didn't catch it

Build kernel from scratch? Not usually - everything is in modules now

What modules are running?
<code>lsmod</code>
* gets info from <code>/proc/modules</code>
* <code>/proc/modules</code> is reading a kernel data structure
* <code>strace lsmod</code>:
** shows that it starts with <code>/proc/modules</code>
** but then goes into <code>/sys/module/</code>
* <code>/proc</code> : human readable
* <code>/sys</code> : machine readable
** wouldn't find a list of modules here, but rather deep directory structure

Where are kernel modules stored?
* <code>/lib/modules/<kernel version>/</code>
* why so many kernel versions?
** ex. 4.15.0-33
*** 4.15.0 is version.
*** -33 = package release (patches etc)

Why so many modules? Why not build them in?
* originally, everything was built in - now we basically never do that
** some may conflict
** many are unnecessary
** memory used by kernel is physically used - never virtual
*** so we minimize this by only loading necessary kernel modules

<code>.ko</code>: kernel object
<code>.so</code>: shared object (ex. C libraries)

Where is kernel itself?
<code>/boot/vmlinuz-4.15.0-33-generic</code>
* naming is traditional. 'z' is for compressed.
* variants may have different suffix, ex. 'rt' for realtime, instead of 'generic'
* kernel is around 8mb

== Review tutorial ==
* <code>sudo insmod simple.ko</code> // load kernel module
* <code>dmesg | tail</code> // shows that kernel is loaded
* <code>sudo rmmod simple.ko</code> // removes kernel module

=== simple.c ===
* Why <code>#include <linux/ ...></code>?
* No headers that are familiar, ex. <code>stdlib</code>, etc.
** You can't use these in kernel modules. Why?
*** You can't use any C library that makes a system call
** What if you wanted to <code>printf</code>?
*** <code>printk()</code> is a reimplementation, because
**** <code>printf</code> would "bottom out" by making a system call to write()
**** <code>printk</code> is used to generate logs
**** how does it do that?
***** keeps a buffer until they can be printed somewhere
***** bootup messages? produced by <code>printk</code>
*** kernel has its own weird world of standard libraries
* Some special syntax around declaring init and exit functions..
* How does kernel actually load a module? (ie. through <code>module_init()</code> call in simple.c)
** don't worry about it. always something you don't understand...
** and this is one of these things few people need to understand!
** you just needs to know how to use it.
* Why doesn't simple_exit return anything? Nobody cares what it returns!
** vs. if init failed... we would care

Rule of kernel programming: do as little as possible
* nicer to write in userspace

=== ones.c ===
* kernel module that implements a device driver for <code>/dev/ones</code>
** from linux kernels perspective.. same as any other device
** a module might be a device driver, but might not be
** and a driver might be baked into the kernel (not a module)
** but usually, device drivers are implemented as modules
* makes a character device: <code>register_chrdev()</code>
** every device has a major and a minor number: the above function returns major number
* <code>ones_fops { }</code>: important. struct containing function pointers.
** we never call these functions in ones.c
** these implement the file operations on <code>/dev/ones</code>
*** after normal syscall dispatcher stuff, it will want to know HOW to open the device, read, etc
*** what if we wanted to do a write?
**** well, <code>/dev/ones</code> is opened read-only. let's <code>chmod a+w /dev/ones</code>.
**** <code>echo 5 > /dev/ones</code> // write error
**** we only have the operations that we've implemented
** who calls them?
* <code>class_create()</code> ?
** can't just create a device from the major number
** need a class first before running <code>device_create()</code>
** grouping of all devices that use the same driver
** see <code>/sys/class/comp3000</code>
* <code>/sys/devices/virtual/comp3000/ones</code>
** what's this about? metadata.
** thankfully we didn't have to create this ourself - <code>device_create()</code> took care of all that
* failed_devreg, etc:
** handling failures is really important in kernel programming.
** if something fails and you don't manually deregister / deallocate memory, it never gets "cleaned up"
*** stays in physical memory until you reboot
* How did we come up with this code, generally?
** no manpages
** the linux kernel source is the ultimate reference for how this works
** best way to figure it out is to look at existing device drivers, modify to your purpose, trial & error...
* <code>ones_read()</code>:
** similar to <code>read()</code> in userspace
** in userspace you take a file descriptor; in kernel you take a pointer to a file struct
** <code>buf, len</code> just as in <code>read()</code>..
** offset? point to specific part of the file
** the file is always going to be <code>/dev/ones</code> in this case (since it is a device driver)
** why <code>put_user('1', buf++)</code> instead of <code>*buf++ = 1</code>?
*** The 1's are going out into userspace. This is what <code>put_user()</code> is for.

aside: <code>find . -name "ones"</code>
* will find files named "ones" in current directory (.)

Look at newgetpid.c

Operating Systems 2018F Lecture 12

2018-11-21T20:40:34Z

R.samanfar: /* Today: Kernel modules */

==Video==

Video from the lecture given on October 17, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec12-20181017.m4a is now available].

==Notes==
{{Please leave this line alone and write below (this is the coloured heading)}}

Today: Kernel modules

=== First, Anil's ssh setup: ===
<code>student@anil-vm:</code> // openstack
* why is it named anil-vm?
** edit /etc/hosts and /etc/hostname
<code>anilclass@sutherland:</code> // local machine
* edit /etc/hosts
** add the IP of your VM and give it name
** distributed DNS does this for URLs
* login without password
** ssh key pair, private and public
** files in ~/.ssh/

Public keys?
* sharing won't compromise security

Running X-windows programs on vm:
* <code>ssh -X student@anil-vm</code> // works natively in linux
* on macOS, you can run X-server separately
* on Windows, use x2go

== Whats a kernel module? ==
* kernel code, that you load, into the kernel
* same privileges as other kernel code
* dangerous! load kernel module = anything can happen
** only superuser (root) can load
* NOT a system call

Kernel Oops:
* kernel catches the bad memory access
Kernel crash:
* didn't catch it

Build kernel from scratch? Not usually - everything is in modules now

What modules are running?
<code>lsmod</code>
* gets info from <code>/proc/modules</code>
* <code>/proc/modules</code> is reading a kernel data structure
* <code>strace lsmod</code>:
** shows that it starts with <code>/proc/modules</code>
** but then goes into <code>/sys/module/</code>
* <code>/proc</code> : human readable
* <code>/sys</code> : machine readable
** wouldn't find a list of modules here, but rather deep directory structure

Where are kernel modules stored?
* <code>/lib/modules/<kernel version>/</code>
* why so many kernel versions?
** ex. 4.15.0-33
*** 4.15.0 is version.
*** -33 = package release (patches etc)

Why so many modules? Why not build them in?
* originally, everything was built in - now we basically never do that
** some may conflict
** many are unnecessary
** memory used by kernel is physically used - never virtual
*** so we minimize this by only loading necessary kernel modules

<code>.ko</code>: kernel object
<code>.so</code>: shared object (ex. C libraries)

Where is kernel itself?
<code>/boot/vmlinuz-4.15.0-33-generic</code>
* naming is traditional. 'z' is for compressed.
* variants may have different suffix, ex. 'rt' for realtime, instead of 'generic'
* kernel is around 8mb

== Review tutorial ==
* <code>sudo insmod simple.ko</code> // load kernel module
* <code>dmesg | tail</code> // shows that kernel is loaded
* <code>sudo rmmod simple.ko</code> // removes kernel module

=== simple.c ===
* Why <code>#include <linux/ ...></code>?
* No headers that are familiar, ex. <code>stdlib</code>, etc.
** You can't use these in kernel modules. Why?
*** You can't use any C library that makes a system call
** What if you wanted to <code>printf</code>?
*** <code>printk()</code> is a reimplementation, because
**** <code>printf</code> would "bottom out" by making a system call to write()
**** <code>printk</code> is used to generate logs
**** how does it do that?
***** keeps a buffer until they can be printed somewhere
***** bootup messages? produced by <code>printk</code>
*** kernel has its own weird world of standard libraries
* Some special syntax around declaring init and exit functions..
* How does kernel actually load a module? (ie. through <code>module_init()</code> call in simple.c)
** don't worry about it. always something you don't understand...
** and this is one of these things few people need to understand!
** you just needs to know how to use it.
* Why doesn't simple_exit return anything? Nobody cares what it returns!
** vs. if init failed... we would care

Rule of kernel programming: do as little as possible
* nicer to write in userspace

=== ones.c ===
* kernel module that implements a device driver for <code>/dev/ones</code>
** from linux kernels perspective.. same as any other device
** a module might be a device driver, but might not be
** and a driver might be baked into the kernel (not a module)
** but usually, device drivers are implemented as modules
* makes a character device: <code>register_chrdev()</code>
** every device has a major and a minor number: the above function returns major number
* <code>ones_fops { }</code>: important. struct containing function pointers.
** we never call these functions in ones.c
** these implement the file operations on <code>/dev/ones</code>
*** after normal syscall dispatcher stuff, it will want to know HOW to open the device, read, etc
*** what if we wanted to do a write?
**** well, <code>/dev/ones</code> is opened read-only. let's <code>chmod a+w /dev/ones</code>.
**** <code>echo 5 > /dev/ones</code> // write error
**** we only have the operations that we've implemented
** who calls them?
* <code>class_create()</code> ?
** can't just create a device from the major number
** need a class first before running <code>device_create()</code>
** grouping of all devices that use the same driver
** see <code>/sys/class/comp3000</code>
* <code>/sys/devices/virtual/comp3000/ones</code>
** what's this about? metadata.
** thankfully we didn't have to create this ourself - <code>device_create()</code> took care of all that
* failed_devreg, etc:
** handling failures is really important in kernel programming.
** if something fails and you don't manually deregister / deallocate memory, it never gets "cleaned up"
*** stays in physical memory until you reboot
* How did we come up with this code, generally?
** no manpages
** the linux kernel source is the ultimate reference for how this works
** best way to figure it out is to look at existing device drivers, modify to your purpose, trial & error...
* <code>ones_read()</code>:
** similar to <code>read()</code> in userspace
** in userspace you take a file descriptor; in kernel you take a pointer to a file struct
** <code>buf, len</code> just as in <code>read()</code>..
** offset? point to specific part of the file
** the file is always going to be <code>/dev/ones</code> in this case (since it is a device driver)
** why <code>put_user('1', buf++)</code> instead of <code>*buf++ = 1</code>?
*** The 1's are going out into userspace. This is what <code>put_user()</code> is for.

aside: <code>find . -name "ones"</code>
* will find files named "ones" in current directory (.)

Look at newgetpid.c

Operating Systems 2018F Lecture 12

2018-11-21T20:39:23Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on October 17, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec12-20181017.m4a is now available].

==Notes==
{{Please leave this line alone and write below (this is the coloured heading)}}

== Today: Kernel modules ==

=== First, Anil's ssh setup: ===
<code>student@anil-vm:</code> // openstack
* why is it named anil-vm?
** edit /etc/hosts and /etc/hostname
<code>anilclass@sutherland:</code> // local machine
* edit /etc/hosts
** add the IP of your VM and give it name
** distributed DNS does this for URLs
* login without password
** ssh key pair, private and public
** files in ~/.ssh/

Public keys?
* sharing won't compromise security

Running X-windows programs on vm:
* <code>ssh -X student@anil-vm</code> // works natively in linux
* on macOS, you can run X-server separately
* on Windows, use x2go

== Whats a kernel module? ==
* kernel code, that you load, into the kernel
* same privileges as other kernel code
* dangerous! load kernel module = anything can happen
** only superuser (root) can load
* NOT a system call

Kernel Oops:
* kernel catches the bad memory access
Kernel crash:
* didn't catch it

Build kernel from scratch? Not usually - everything is in modules now

What modules are running?
<code>lsmod</code>
* gets info from <code>/proc/modules</code>
* <code>/proc/modules</code> is reading a kernel data structure
* <code>strace lsmod</code>:
** shows that it starts with <code>/proc/modules</code>
** but then goes into <code>/sys/module/</code>
* <code>/proc</code> : human readable
* <code>/sys</code> : machine readable
** wouldn't find a list of modules here, but rather deep directory structure

Where are kernel modules stored?
* <code>/lib/modules/<kernel version>/</code>
* why so many kernel versions?
** ex. 4.15.0-33
*** 4.15.0 is version.
*** -33 = package release (patches etc)

Why so many modules? Why not build them in?
* originally, everything was built in - now we basically never do that
** some may conflict
** many are unnecessary
** memory used by kernel is physically used - never virtual
*** so we minimize this by only loading necessary kernel modules

<code>.ko</code>: kernel object
<code>.so</code>: shared object (ex. C libraries)

Where is kernel itself?
<code>/boot/vmlinuz-4.15.0-33-generic</code>
* naming is traditional. 'z' is for compressed.
* variants may have different suffix, ex. 'rt' for realtime, instead of 'generic'
* kernel is around 8mb

== Review tutorial ==
* <code>sudo insmod simple.ko</code> // load kernel module
* <code>dmesg | tail</code> // shows that kernel is loaded
* <code>sudo rmmod simple.ko</code> // removes kernel module

=== simple.c ===
* Why <code>#include <linux/ ...></code>?
* No headers that are familiar, ex. <code>stdlib</code>, etc.
** You can't use these in kernel modules. Why?
*** You can't use any C library that makes a system call
** What if you wanted to <code>printf</code>?
*** <code>printk()</code> is a reimplementation, because
**** <code>printf</code> would "bottom out" by making a system call to write()
**** <code>printk</code> is used to generate logs
**** how does it do that?
***** keeps a buffer until they can be printed somewhere
***** bootup messages? produced by <code>printk</code>
*** kernel has its own weird world of standard libraries
* Some special syntax around declaring init and exit functions..
* How does kernel actually load a module? (ie. through <code>module_init()</code> call in simple.c)
** don't worry about it. always something you don't understand...
** and this is one of these things few people need to understand!
** you just needs to know how to use it.
* Why doesn't simple_exit return anything? Nobody cares what it returns!
** vs. if init failed... we would care

Rule of kernel programming: do as little as possible
* nicer to write in userspace

=== ones.c ===
* kernel module that implements a device driver for <code>/dev/ones</code>
** from linux kernels perspective.. same as any other device
** a module might be a device driver, but might not be
** and a driver might be baked into the kernel (not a module)
** but usually, device drivers are implemented as modules
* makes a character device: <code>register_chrdev()</code>
** every device has a major and a minor number: the above function returns major number
* <code>ones_fops { }</code>: important. struct containing function pointers.
** we never call these functions in ones.c
** these implement the file operations on <code>/dev/ones</code>
*** after normal syscall dispatcher stuff, it will want to know HOW to open the device, read, etc
*** what if we wanted to do a write?
**** well, <code>/dev/ones</code> is opened read-only. let's <code>chmod a+w /dev/ones</code>.
**** <code>echo 5 > /dev/ones</code> // write error
**** we only have the operations that we've implemented
** who calls them?
* <code>class_create()</code> ?
** can't just create a device from the major number
** need a class first before running <code>device_create()</code>
** grouping of all devices that use the same driver
** see <code>/sys/class/comp3000</code>
* <code>/sys/devices/virtual/comp3000/ones</code>
** what's this about? metadata.
** thankfully we didn't have to create this ourself - <code>device_create()</code> took care of all that
* failed_devreg, etc:
** handling failures is really important in kernel programming.
** if something fails and you don't manually deregister / deallocate memory, it never gets "cleaned up"
*** stays in physical memory until you reboot
* How did we come up with this code, generally?
** no manpages
** the linux kernel source is the ultimate reference for how this works
** best way to figure it out is to look at existing device drivers, modify to your purpose, trial & error...
* <code>ones_read()</code>:
** similar to <code>read()</code> in userspace
** in userspace you take a file descriptor; in kernel you take a pointer to a file struct
** <code>buf, len</code> just as in <code>read()</code>..
** offset? point to specific part of the file
** the file is always going to be <code>/dev/ones</code> in this case (since it is a device driver)
** why <code>put_user('1', buf++)</code> instead of <code>*buf++ = 1</code>?
*** The 1's are going out into userspace. This is what <code>put_user()</code> is for.

aside: <code>find . -name "ones"</code>
* will find files named "ones" in current directory (.)

Look at newgetpid.c

Operating Systems 2018F Lecture 11

2018-11-21T20:38:49Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on October 10, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec11-20181010.m4a is now available].

==Notes==
* Today:
** Discussing Assignment Solutions

* From Assignment 2:
** The first two questions are related, however, the first question requires you make a hard link. A hard link means that both associations point to the same inodes. A hard link just creates a separate differently named association that accesses the same file. (Hard links only work within a single file system)
** The second question involves creating a symbolic link. Running lstat() returns the inode relating to the link instead of to the file.
** Question 3 statbuf has the user id and group id stored. The username and group name can be looked up via the password file using getgrgrid() and getpwuid().
** Question 4 Trying to write to the file produces a SEGFAULT. The reason for this is that you do not have access to write to the file as it is readonly. A SEGFAULT will be produced any time an attempt is made to access private memory. PROT_READ would allow you to have access however.
** Question 6 Placing an exit command within the if statement at line 215 will permanently stop the consumer.
** Question 7 malloc uses private memory whereas mmap does not. Replacing mmap with malloc means the producer and consumer will be accessing separate memory and the consumer will never consume.
** Qestion 8 The kill signals are interrupting the sleep commands. Removing the kill signals means the sleeps will never be interrupted and there will be a delay every time.
** Question 9 You are not supposed to be able to predict the output of /dev/random. Technically you can if you can figure out the inputs. If the input was compromised then the random numbers could be compromised.
** Question 10 sem_wait() and sem_post() do not produce system calls. They do require special CPU instructions, but do not need to make system calls.

* Why use goto?
** In C, there isnt built in error handling such as try catch, so goto can be used for error handling since they stay in method scope.
** Linux kernel code uses goto in this manner.

* Assignment 1
** Question 1 When a process is run in the foreground the shell will wait for the program to terminate, but when a process is run in the background the shell will not wait. This means the default is to run background processes and extra information is required to monitor processes in the foreground.
** Question 2 Signals are not received by system calls.
** Question 4 Removing the wait() command will prevent the child processes from being properly terminated and so they will accumulate as zombie processes.
** Question 9 execve does not close file descriptors. If it did you couldnt have stdIn, stdError. The file descriptors for stdOut is 0, stdIn is 1, and stdError is 2. File descriptors are what are returned when a file is opened for the purposes of reading and writing to the file. Therefore, every open file will have a file descriptor.
** Question 10 An in-progress system call is interrupted upon receiving a signal. It can possibly be restarted later.
** Question 11 plist is doing an ls command on /proc. This specifically shows comm files which most other directories will not contain. Thus giving different directories to plist will not give much back.
** Question 12 Input the file through $echo FileToDirect > shell3000. Use dup2 to retrieve the file descriptor and then run it using execve.

Operating Systems 2018F Lecture 8

2018-10-02T00:00:33Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 26, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec08-20180928.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec08/ here].

==Notes==

*Things you can do with remote linux server:
** '''ssh''' into the server and use the command line
** using `x2goclient` can have remote desktop access

*The “cloud” just means computer infrastructure. The “cloud” is really just someone else computer.

*'''scp''' : secure copy (remote file copy program)

*Download `WinSCP` for remote file transfers on windows.
*On macOS or Linux use `scp` for remote file transfer.

*How do you do backups?
**'''rsync''': a fast, versatile, remote (and local) file-copying tool

[[File:Lec8-1.png|none|800px]]

*Send a file or directory to an address:

[[File:Lec8-2.png|none|800px]]
[[File:Lec8-3.png|none|800px]]

*Writing an alias for '''rsync''' in '''~/.bashrc''':

[[File:Lec8-4.png|none|800px]]

*'''mmap''', '''munmap''' - map or unmap files or devices into memory

[[File:Lec8-5.png|none|800px]]

*'''mmap''' allows us to allocate memory to hold the contents of the file in a memory efficient way. It does this _lazily_ meaning that it only loads into memory what is being read, at the time that it is being read.

File:Lec8-4.png

2018-10-01T23:59:04Z

R.samanfar: R.samanfar uploaded a new version of File:Lec8-4.png

Operating Systems 2018F Lecture 8

2018-10-01T23:57:22Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 26, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec08-20180928.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec08/ here].

==Notes==

*Things you can do with remote linux server:
** '''ssh''' into the server and use the command line
** using `x2goclient` can have remote desktop access

*The “cloud” just means computer infrastructure. The “cloud” is really just someone else computer.

*'''scp''' : secure copy (remote file copy program)

*Download `WinSCP` for remote file transfers on windows.
*On macOS or Linux use `scp` for remote file transfer.

*How do you do backups?
**'''rsync''': a fast, versatile, remote (and local) file-copying tool
[[File:Lec8-1.png|none|800px]]

*Send a file or directory to an address:
[[File:Lec8-2.png|none|800px]]
[[File:Lec8-3.png|none|800px]]

*Writing an alias for '''rsync''' in '''~/.bashrc''':
[[File:Lec8-4.png|none|800px]]

*'''mmap''', '''munmap''' - map or unmap files or devices into memory

[[File:Lec8-5.png|none|800px]]

*'''mmap''' allows us to allocate memory to hold the contents of the file in a memory efficient way. It does this _lazily_ meaning that it only loads into memory what is being read, at the time that it is being read.

File:Lec8-5.png

2018-10-01T23:55:13Z

R.samanfar:

File:Lec8-4.png

2018-10-01T23:55:00Z

R.samanfar:

File:Lec8-3.png

2018-10-01T23:54:52Z

R.samanfar:

File:Lec8-2.png

2018-10-01T23:54:44Z

R.samanfar:

File:Lec8-1.png

2018-10-01T23:54:35Z

R.samanfar:

Operating Systems 2018F Lecture 8

2018-10-01T23:54:18Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 26, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec08-20180928.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec08/ here].

==Notes==

*Things you can do with remote linux server:
** '''ssh''' into the server and use the command line
** using `x2goclient` can have remote desktop access

*The “cloud” just means computer infrastructure. The “cloud” is really just someone else computer.

*'''scp''' : secure copy (remote file copy program)

*Download `WinSCP` for remote file transfers on windows.
*On macOS or Linux use `scp` for remote file transfer.

*How do you do backups?
**'''rsync''': a fast, versatile, remote (and local) file-copying tool
*Send a file or directory to an address:

*Writing an alias for '''rsync''' in '''~/.bashrc''':

*'''mmap''', '''munmap''' - map or unmap files or devices into memory

*'''mmap''' allows us to allocate memory to hold the contents of the file in a memory efficient way. It does this _lazily_ meaning that it only loads into memory what is being read, at the time that it is being read.

Operating Systems 2018F Lecture 6

2018-10-01T23:36:19Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 21, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec06-20180921.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec06/ here].

==Notes==

===Terminal Notes:===

*ls -la →
1st column: read/write permissions. (r: read, w: write, x: execute)
2nd column: # of pointers to a file/directory.
3rd column: user (owner)
4th column: group
5th column: size of file/directory
6th column: date last modified
Last column: file/directory name

*ls -lai →
list the inodes as well (shows in first column)

*ln -s f1 f2 →
Create symbolic link. f2 becomes a reference to f1.

'''Note:''' f2 is refering to f1's name. NOT its inode. (f1 and f2 are the file/directory names)
If you remove f1 after creating the above link, it will break the link.

*ln f1 f3 →
Create hard link. f3 becomes a reference to f1.

Note: The absence of the '-s' in the ln command causes the link to be to f1's inode. If you remove f1 after creating the above link, it will NOT break the link.

*More Information on Symbolic Links and Inode's:
**A symbolic link that refers to a file that doesn't exist is broken. You have to recreate the target to fix it.
**Symbolic link can link to a file OR a directory.
**They can also point to filesystems.
**Inode #s are only unique to the specific filesystem they belong to.
'''Example:''' For a given file in your user directory... a file with an identical inode can exist in the virtual filesystem (/proc).
*Information on removing files:
**rm in Unix does not necessarily remove the file. It only removes the name and space.
**The inode is only reclaimed when there are no references remaining for the file.
*Information on finding inodes:
**Only the kernel can give us information about a file name's associated inode. Therefore, we must make a system call. (i.e. lstat, fstat, etc..)
**ls -l → the '-l' option does a 'stat' system call on the directory to also provide file info such as name, size, etc..

'''Note:''' lsat will return information about a link if a symbolic link exists, rather than the file that is being linked to. Fsat will not do this. Fsat will give information about the linked file.
*Information on Filesystems:
**Linux supports many filesystems, whereas windows does not.
**df . → tells you the location of the filesystem of the current directory you are in.
===Lecture Notes===
*inode
**key data structure in UNIX-like filesystems

*Files have two parts
**file inode (metadata)
**file data
*Directories associate file names with inodes (specifically, inode numbers)

*file metadata in an inode
**owner
**group
**size
**permissions
**pointers to data blocks (in some format)

*Where are filesystems stored?
**persistent arrays
**array entries are "blocks"
**4K traditionally, but can be bigger or smaller

*To access, kernel gives device a block number, gets back block contents.

Operating Systems 2018F Lecture 5

2018-10-01T23:14:01Z

R.samanfar:

==Video==

Video from the lecture given on September 19, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec05-20180919.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec05/ here].

==Notes==

===Notes on Tutorial 2===

'''Q6:''' Make the shell output "Ouch!" when you send it a SIGUSR1 signal.
*To test, send a signal to a process
*Use kill command
**kill -<SIGNALNAME> <PID>
**ex: kill -USR1 4094
'''Q8:''' Replace the use of find_env() with getenv(). How do their interfaces differ?
*How are the parameters/arguments different
*Look at man pages for getenv()
*Different number of arguments
*Why does getenv() not need the environment to be passed in
** It has it already
**#man pages, see also, environ is user environment global variable defined by standard library
**#Longer answer: we got the environment variable from main as an argument (*envp[])
'''Q9:''' Use putenv() to implement LASTCOMMAND from Tutorial 1.
*putenv() handles variables for you
*Allows you to define environment variables and add it
*Only changes for current process
**To change for another process, fork and use execve
===Notes on Assignment 1 ===
*Not finalized = “not completely debugged”
*Due Sept 26th 2:30pm
*Based on tutorials
*Question 9 looking at man page for execve can get an answer- not what he’s talking about, he wants experimental evidence
*Solutions will be posted and discussed in class Sept 26th
===Class Notes===

*Signals:
**is a message sent to a process
**some can be blocked until unblocked
**SIGKILL and SIGSTOP cannot be caught, blocked or ignored
**Handlers respond to specific signals
***Interrupts what’s currently happening
***Can cause jumps that happen randomly in your program
***You don’t know the state of the program when a signal handler executes
***Kernel calls a function
***Don’t try to handle a signal in a signal handler, generally block them in a handler
***If you don’t define a signal handler, there is a default handler that will run
**No signal handler for SIGKILL or SIGSTOP
**Can run strace on kill signal
***Does kill system call
***Takes a process id and a signal
*I/O redirection:
**changing which file things get written to
**ls > foo.txt
***foo.txt contains the contents that ls outputted
***'''>''' send output to foo.txt
***ls does not know about redirection
***ls should send it is output to foo.txt
***ls without redirection is already outputting to a file (standard output- file descriptor 1)
****standard out sends information to a file
****fprintf – goes to file descriptor 2
****printf – goes to file descriptor 1
**ps aux
***shows you what’s running and what terminal it is connected to (tty2, ?, pts/0)
****pts/0 – sudo terminal
****tty1 – used to run log in
***terminal usually referred to as tty (teletype, screen and keyboard, connects to another computer)
*File descriptors
**a number associated with an open file
**each process has its own file descriptor
**first file opened is assigned file descriptor 0
**stored on the kernel
**by default in linux there are file descriptors for every program
**the shell makes sure that file descriptors 0,1,2 are open and are going somewhere valid
===Useful Commands/examples===

*envp – came from main, can be changed by putenv
*ls /proc
**Tells you information on all the running processes
**Can cd into a process id directory
***fd – file descriptor directory
***ex: /proc/1973/fd
**echo $TERM
**tells you what terminal you are using
**TERM is an environment variable
*cd /dev
**can see all of the special files
**b – block devices
**c – character devices
*echo hello > 0
*echo hello > 1
**run in one terminal, can write to file displayed in another terminal
**numbers correspond to different terminals being used
**if you have permission you can write to them (other users would not be allowed)
*echo hello > test1
**to the shell, please open the file test1 for writing, open it on file descriptor 1, the process will then do an execve of echo
**cat test -> results in “hello”
*cat 333shell.c | sort
**vertical bar is a pipe
**whatever is written by cat will be read by sort
**pairing of file descriptors
**sorted all the lines of the program
*pstree
**displays the tree of all running processes

Operating Systems 2018F Lecture 5

2018-10-01T23:12:33Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 19, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec05-20180919.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec05/ here].

==Notes==
===Notes on Tutorial 2===
'''Q6:''' Make the shell output "Ouch!" when you send it a SIGUSR1 signal.
*To test, send a signal to a process
*Use kill command
**kill -<SIGNALNAME> <PID>
**ex: kill -USR1 4094
'''Q8:''' Replace the use of find_env() with getenv(). How do their interfaces differ?
*How are the parameters/arguments different
*Look at man pages for getenv()
*Different number of arguments
*Why does getenv() not need the environment to be passed in
** It has it already
**#man pages, see also, environ is user environment global variable defined by standard library
**#Longer answer: we got the environment variable from main as an argument (*envp[])
'''Q9:''' Use putenv() to implement LASTCOMMAND from Tutorial 1.
*putenv() handles variables for you
*Allows you to define environment variables and add it
*Only changes for current process
**To change for another process, fork and use execve
===Notes on Assignment 1 ===
*Not finalized = “not completely debugged”
*Due Sept 26th 2:30pm
*Based on tutorials
*Question 9 looking at man page for execve can get an answer- not what he’s talking about, he wants experimental evidence
*Solutions will be posted and discussed in class Sept 26th
===Class Notes===
*Signals:
**is a message sent to a process
**some can be blocked until unblocked
**SIGKILL and SIGSTOP cannot be caught, blocked or ignored
**Handlers respond to specific signals
***Interrupts what’s currently happening
***Can cause jumps that happen randomly in your program
***You don’t know the state of the program when a signal handler executes
***Kernel calls a function
***Don’t try to handle a signal in a signal handler, generally block them in a handler
***If you don’t define a signal handler, there is a default handler that will run
**No signal handler for SIGKILL or SIGSTOP
**Can run strace on kill signal
***Does kill system call
***Takes a process id and a signal
*I/O redirection:
**changing which file things get written to
**ls > foo.txt
***foo.txt contains the contents that ls outputted
***'''>''' send output to foo.txt
***ls does not know about redirection
***ls should send it is output to foo.txt
***ls without redirection is already outputting to a file (standard output- file descriptor 1)
****standard out sends information to a file
****fprintf – goes to file descriptor 2
****printf – goes to file descriptor 1
**ps aux
***shows you what’s running and what terminal it is connected to (tty2, ?, pts/0)
****pts/0 – sudo terminal
****tty1 – used to run log in
***terminal usually referred to as tty (teletype, screen and keyboard, connects to another computer)
*File descriptors
**a number associated with an open file
**each process has its own file descriptor
**first file opened is assigned file descriptor 0
**stored on the kernel
**by default in linux there are file descriptors for every program
**the shell makes sure that file descriptors 0,1,2 are open and are going somewhere valid
===Useful Commands/examples===
*envp – came from main, can be changed by putenv
*ls /proc
**Tells you information on all the running processes
**Can cd into a process id directory
***fd – file descriptor directory
***ex: /proc/1973/fd
**echo $TERM
**tells you what terminal you are using
**TERM is an environment variable
*cd /dev
**can see all of the special files
**b – block devices
**c – character devices
*echo hello > 0
*echo hello > 1
**run in one terminal, can write to file displayed in another terminal
**numbers correspond to different terminals being used
**if you have permission you can write to them (other users would not be allowed)
*echo hello > test1
**to the shell, please open the file test1 for writing, open it on file descriptor 1, the process will then do an execve of echo
**cat test -> results in “hello”
*cat 333shell.c | sort
**vertical bar is a pipe
**whatever is written by cat will be read by sort
**pairing of file descriptors
**sorted all the lines of the program
*pstree
**displays the tree of all running processes

Operating Systems 2018F Lecture 3

2018-10-01T22:51:26Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 12, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec03-20180912.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec03/ here].

==Notes==
*Sources of confusion from Tutorial 1:
**Where is good assembly documentation?
***Probably in books
***Online
***(Not expected to learn assembly)
**Local variables
**Environment variables
**gcc arguments
*The –O2 argument for gcc is for choosing an optimization level (namely, level 2)
===Linking===
*A file that is statically linked does not have any library calls
*What is linking?
**Taking object files and putting them together to make a full binary, particularly resolving symbolic references between object files.
*Static linking is the old way. Dynamic linking is how we do things today.
*nm command for showing symbols
===System and Function Calls===
*What is a system call in assembly? What is a function call in assembly?
*(Time: 9:00) Looking at hello.c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello world!\n");
return 0;
}
*Professor enters the command: gcc –S –O2 hello.c, which generates the assembly file hello.s . The –S option for gcc does not tell it to do more work, it tells it to do less work.

*(Time: 12:05) Looking at the assembly file of hello.c
[[File:Lec3-1.png|none|800px]]
**All of the things with dot (.) in front of them are declarations (instructions to the assembler) and are not actual operations.
**Looking at the line that has a function call to puts. Since we are not using much of the fancy functionality of printf (the function we actually called), a modern compiler environment does optimization which involves changing code and swapping a fancy function like printf with a simpler function like puts to print what we want to the screen.
**The name for the string (i.e. “Hello world!”) is .LC0
**Compilation is a black box
**The call operation in assembly is for calling functions. The puts that we see is a function call.
**System calls are fundamentally different from function calls. They are different in a way
*such that they cannot be created in standard C.
**We can create a system call in only two ways:
***putting inline assembly in standard C code.
***use a compiler specific operation that tells it to generate code for the system call.
*Professor writes the following commands on the terminal:
**gcc –static –O2 hello.c –o hello
**strace ./hello à strace gives you the system calls that a running program makes.
**(Time: 26:25) The write system call is the system call that actually did the work that we intended to do.
**So, the puts function somehow made a write system call which is what produced the text output to the console.
*What is a function call? The following happens when you call a function:
**Save arguments (to registers or the stack)
**Call function
***Saves (pushes) current instruction pointer onto call stack
***Changes instruction pointer to the location of the beginning of the function
***... Function runs...
***on function return, pops instruction pointer from the stack, restoring the old function pointer.
*Everything is based on pointers to code
*'''(Time: 35:00)''' The nm command tells us, in the first column of its output, where things are located.
**If you look at an object file with nm, before it has been linked, you will see that all addresses are zero. This means that we do not know where they are in memory.
**In contrast, when we look at a compiled program we can see symbols at specific addresses in memory.
**Are these addresses unique to this program? In other words, is any particular address, in the program symbol table listed, unique across all programs running on my computer?
***No!
***This means that you could have a function located in the same memory location in completely different processes.
***Pointers in a process are '''local'''
***Addresses only have meaning in the context of the process
***Every process has a completely different namespace
*Address space – name space for pointers.
*(Time : 39:00) What is a System Call?
**It is an invocation of kernel code.
**The kernel is the code that defines the system. It is what implements the processes. It implements the namespaces that make separate processes.
**A system call entails calling code that is not necessarily in your address space.
**A system call entails calling code with higher privileges.
***Running in supervisor mode, not user mode
**How can I call privileged code safely? (code that runs in supervisor mode)
***Restrict the entry points to privileged code
****Can’t call arbitrary routines.
***Entry point should check whether operation is allowed.
***In the kernel, this is what the system call dispatcher does
**System Call Dispatcher
***Process requests system call (write)
***CPU switches to supervisor mode, runs system call dispatcher
***Dispatcher decides if system call is allowed
***System call code (write) is invoked
**CPU switches into supervisor mode using special instructions
***“software interrupts”
***“upcall” → How do I call the kernel
**The C library has function wrappers around standard system calls.
**When we say that we are making a ‘system call’ from C, we normally mean that we are making a function call, whose function is a wrapper around the system call we intended to call.
**Professor enters the following commands:
***gcc –O2 hello.c –o hello
***ltrace ./hello
***ltrace tells you about functions that are called. After this command, we see that ltrace says that there is a puts function.
***strace ./hello
***After this command, we can see lots of system calls. We can identify the write system call at the end.
***gcc –O2 –static hello.c –o hello
***strace ./hello
***When linking statically, we can see that we get a lot less system calls.
*At the function call level, you call code inside your process.
*At the system call level, you call code outside your process. System calls are more expensive than function calls.
*We have not seen any assembly of a system call yet.
===Walking through CSimpleShell.c===
*'''(Time: 1:00:13)''' Looking at csimpleshell.c
[[File:Lec3-2.png|none|800px]]
*We have an infinite loop in which we do:
**Print ‘$’ sign, for printing a prompt
**fgets is for reading an input
**parse_args → parses arguments, converts a single string into an array of strings
***takes the buffer of the input from the user
***takes in args which is an array of ARR_SIZE (which is 1<<16 → means 1 right shifted 16 bits i.e. 2^(16)).
***Takes in nargs which is the number of arguments
If there are no args (nargs == 0), then we continue (i.e. go to the next iteration of
the loop).
**If the first argument == “exit”, then call the exit function
**Otherwise, we fork with the function fork( )
**If the pid is not 0, then the current process is the parent process. It prints that it is waiting for the child , waits, and then prints that the child is done.
**If the pid is 0, the current process is the child process. The child process calls execvp which is like execve, except that you don’t need to supply the environment variables (it just needs the arguments).
**If execvp fails, the puts function below it will be called to indicate an error on the terminal and we exit with exit(127
*We can run external programs with execve
*Every process has its own namespace, memory, address space. How then, do you pass variables from one to the other?
*Where is a username stored? Is a system call required to get the value of username?
**Username information is inside the process. It is in the address space. A system call is not required to get the value of username.
**Professor prints the environment variables.
**When you load a program binary you give execve:
*#The binary
*#The command line arguments
*#Environment variables
**Environment variables are available as an argument to main. They are variables that are in your process that the kernel happens to put there when you start.

File:Lec3-2.png

2018-10-01T22:50:11Z

R.samanfar:

File:Lec3-1.png

2018-10-01T22:50:02Z

R.samanfar:

Operating Systems 2018F Lecture 3

2018-10-01T22:49:38Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 12, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec03-20180912.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec03/ here].

==Notes==
*Sources of confusion from Tutorial 1:
**Where is good assembly documentation?
***Probably in books
***Online
***(Not expected to learn assembly)
**Local variables
**Environment variables
**gcc arguments
*The –O2 argument for gcc is for choosing an optimization level (namely, level 2)
===Linking===
*A file that is statically linked does not have any library calls
*What is linking?
**Taking object files and putting them together to make a full binary, particularly resolving symbolic references between object files.
*Static linking is the old way. Dynamic linking is how we do things today.
*nm command for showing symbols
===System and Function Calls===
*What is a system call in assembly? What is a function call in assembly?
*(Time: 9:00) Looking at hello.c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello world!\n");
return 0;
}
*Professor enters the command: gcc –S –O2 hello.c, which generates the assembly file hello.s . The –S option for gcc does not tell it to do more work, it tells it to do less work.

*(Time: 12:05) Looking at the assembly file of hello.c
**All of the things with dot (.) in front of them are declarations (instructions to the assembler) and are not actual operations.
**Looking at the line that has a function call to puts. Since we are not using much of the fancy functionality of printf (the function we actually called), a modern compiler environment does optimization which involves changing code and swapping a fancy function like printf with a simpler function like puts to print what we want to the screen.
**The name for the string (i.e. “Hello world!”) is .LC0
**Compilation is a black box
**The call operation in assembly is for calling functions. The puts that we see is a function call.
**System calls are fundamentally different from function calls. They are different in a way
*such that they cannot be created in standard C.
**We can create a system call in only two ways:
***putting inline assembly in standard C code.
***use a compiler specific operation that tells it to generate code for the system call.
*Professor writes the following commands on the terminal:
**gcc –static –O2 hello.c –o hello
**strace ./hello à strace gives you the system calls that a running program makes.
**(Time: 26:25) The write system call is the system call that actually did the work that we intended to do.
**So, the puts function somehow made a write system call which is what produced the text output to the console.
*What is a function call? The following happens when you call a function:
**Save arguments (to registers or the stack)
**Call function
***Saves (pushes) current instruction pointer onto call stack
***Changes instruction pointer to the location of the beginning of the function
***... Function runs...
***on function return, pops instruction pointer from the stack, restoring the old function pointer.
*Everything is based on pointers to code
*'''(Time: 35:00)''' The nm command tells us, in the first column of its output, where things are located.
**If you look at an object file with nm, before it has been linked, you will see that all addresses are zero. This means that we do not know where they are in memory.
**In contrast, when we look at a compiled program we can see symbols at specific addresses in memory.
**Are these addresses unique to this program? In other words, is any particular address, in the program symbol table listed, unique across all programs running on my computer?
***No!
***This means that you could have a function located in the same memory location in completely different processes.
***Pointers in a process are '''local'''
***Addresses only have meaning in the context of the process
***Every process has a completely different namespace
*Address space – name space for pointers.
*(Time : 39:00) What is a System Call?
**It is an invocation of kernel code.
**The kernel is the code that defines the system. It is what implements the processes. It implements the namespaces that make separate processes.
**A system call entails calling code that is not necessarily in your address space.
**A system call entails calling code with higher privileges.
***Running in supervisor mode, not user mode
**How can I call privileged code safely? (code that runs in supervisor mode)
***Restrict the entry points to privileged code
****Can’t call arbitrary routines.
***Entry point should check whether operation is allowed.
***In the kernel, this is what the system call dispatcher does
**System Call Dispatcher
***Process requests system call (write)
***CPU switches to supervisor mode, runs system call dispatcher
***Dispatcher decides if system call is allowed
***System call code (write) is invoked
**CPU switches into supervisor mode using special instructions
***“software interrupts”
***“upcall” → How do I call the kernel
**The C library has function wrappers around standard system calls.
**When we say that we are making a ‘system call’ from C, we normally mean that we are making a function call, whose function is a wrapper around the system call we intended to call.
**Professor enters the following commands:
***gcc –O2 hello.c –o hello
***ltrace ./hello
***ltrace tells you about functions that are called. After this command, we see that ltrace says that there is a puts function.
***strace ./hello
***After this command, we can see lots of system calls. We can identify the write system call at the end.
***gcc –O2 –static hello.c –o hello
***strace ./hello
***When linking statically, we can see that we get a lot less system calls.
*At the function call level, you call code inside your process.
*At the system call level, you call code outside your process. System calls are more expensive than function calls.
*We have not seen any assembly of a system call yet.
===Walking through CSimpleShell.c===
*'''(Time: 1:00:13)''' Looking at csimpleshell.c
*We have an infinite loop in which we do:
**Print ‘$’ sign, for printing a prompt
**fgets is for reading an input
**parse_args → parses arguments, converts a single string into an array of strings
***takes the buffer of the input from the user
***takes in args which is an array of ARR_SIZE (which is 1<<16 → means 1 right shifted 16 bits i.e. 2^(16)).
***Takes in nargs which is the number of arguments
If there are no args (nargs == 0), then we continue (i.e. go to the next iteration of
the loop).
**If the first argument == “exit”, then call the exit function
**Otherwise, we fork with the function fork( )
**If the pid is not 0, then the current process is the parent process. It prints that it is waiting for the child , waits, and then prints that the child is done.
**If the pid is 0, the current process is the child process. The child process calls execvp which is like execve, except that you don’t need to supply the environment variables (it just needs the arguments).
**If execvp fails, the puts function below it will be called to indicate an error on the terminal and we exit with exit(127
*We can run external programs with execve
*Every process has its own namespace, memory, address space. How then, do you pass variables from one to the other?
*Where is a username stored? Is a system call required to get the value of username?
**Username information is inside the process. It is in the address space. A system call is not required to get the value of username.
**Professor prints the environment variables.
**When you load a program binary you give execve:
*#The binary
*#The command line arguments
*#Environment variables
**Environment variables are available as an argument to main. They are variables that are in your process that the kernel happens to put there when you start.

Operating Systems 2018F Lecture 3

2018-10-01T22:39:50Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 12, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec03-20180912.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec03/ here].

==Notes==
*Sources of confusion from Tutorial 1:
**Where is good assembly documentation?
***Probably in books
***Online
***(Not expected to learn assembly)
**Local variables
**Environment variables
**gcc arguments
*The –O2 argument for gcc is for choosing an optimization level (namely, level 2)
===Linking===
*A file that is statically linked does not have any library calls
*What is linking?
**Taking object files and putting them together to make a full binary, particularly resolving symbolic references between object files.
*Static linking is the old way. Dynamic linking is how we do things today.
*nm command for showing symbols
===System and Function Calls===
*What is a system call in assembly? What is a function call in assembly?
*(Time: 9:00) Looking at hello.c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello world!\n");
return 0;
}
*Professor enters the command: gcc –S –O2 hello.c, which generates the assembly file hello.s . The –S option for gcc does not tell it to do more work, it tells it to do less work.

*(Time: 12:05) Looking at the assembly file of hello.c
**All of the things with dot (.) in front of them are declarations (instructions to the assembler) and are not actual operations.
**Looking at the line that has a function call to puts. Since we are not using much of the fancy functionality of printf (the function we actually called), a modern compiler environment does optimization which involves changing code and swapping a fancy function like printf with a simpler function like puts to print what we want to the screen.
**The name for the string (i.e. “Hello world!”) is .LC0
**Compilation is a black box
**The call operation in assembly is for calling functions. The puts that we see is a function call.
**System calls are fundamentally different from function calls. They are different in a way
*such that they cannot be created in standard C.
**We can create a system call in only two ways:
***putting inline assembly in standard C code.
***use a compiler specific operation that tells it to generate code for the system call.
*Professor writes the following commands on the terminal:
**gcc –static –O2 hello.c –o hello
**strace ./hello à strace gives you the system calls that a running program makes.
**(Time: 26:25) The write system call is the system call that actually did the work that we intended to do.
**So, the puts function somehow made a write system call which is what produced the text output to the console.
*What is a function call? The following happens when you call a function:
**Save arguments (to registers or the stack)
**Call function
***Saves (pushes) current instruction pointer onto call stack
***Changes instruction pointer to the location of the beginning of the function
***... Function runs...
***on function return, pops instruction pointer from the stack, restoring the old function pointer.
*Everything is based on pointers to code
*'''(Time: 35:00)''' The nm command tells us, in the first column of its output, where things are located.
**If you look at an object file with nm, before it has been linked, you will see that all addresses are zero. This means that we do not know where they are in memory.
**In contrast, when we look at a compiled program we can see symbols at specific addresses in memory.
**Are these addresses unique to this program? In other words, is any particular address, in the program symbol table listed, unique across all programs running on my computer?
***No!
***This means that you could have a function located in the same memory location in completely different processes.
***Pointers in a process are '''local'''
***Addresses only have meaning in the context of the process
***Every process has a completely different namespace
*Address space – name space for pointers.
*(Time : 39:00) What is a System Call?
**It is an invocation of kernel code.
**The kernel is the code that defines the system. It is what implements the processes. It implements the namespaces that make separate processes.
**A system call entails calling code that is not necessarily in your address space.
**A system call entails calling code with higher privileges.
***Running in supervisor mode, not user mode
**How can I call privileged code safely? (code that runs in supervisor mode)
***Restrict the entry points to privileged code
****Can’t call arbitrary routines.
***Entry point should check whether operation is allowed.
***In the kernel, this is what the system call dispatcher does
**System Call Dispatcher
***Process requests system call (write)
***CPU switches to supervisor mode, runs system call dispatcher
***Dispatcher decides if system call is allowed
***System call code (write) is invoked
**CPU switches into supervisor mode using special instructions
***“software interrupts”
***“upcall” → How do I call the kernel
**The C library has function wrappers around standard system calls.
**When we say that we are making a ‘system call’ from C, we normally mean that we are making a function call, whose function is a wrapper around the system call we intended to call.
**Professor enters the following commands:
***gcc –O2 hello.c –o hello
***ltrace ./hello
***ltrace tells you about functions that are called. After this command, we see that ltrace says that there is a puts function.
***strace ./hello
***After this command, we can see lots of system calls. We can identify the write system call at the end.
***gcc –O2 –static hello.c –o hello
***strace ./hello
***When linking statically, we can see that we get a lot less system calls.
*At the function call level, you call code inside your process.
*At the system call level, you call code outside your process. System calls are more expensive than function calls.
*We have not seen any assembly of a system call yet.

Operating Systems 2018F Lecture 2

2018-09-19T17:09:21Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 7, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec02-20180907.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec02/ here].

==Notes==
'''Where do processes come from?'''
* Can be visualized as a tree hierarchy
* Every process has a parent and a child
* In windows one process specifies the executable of another

'''Orthogonal Design in API's: Unix Principle'''
* Only one way of doing an operation
* Complex operations are made by simple ones

'''Process Duplication'''

Cases for making a new process:
* Run a new program
* Duplicate an existing process

'''Why would you think of duplicating a process?'''
* It's not the same as running a process a second time
** Variables and state are carried over
* The duplicate becomes the child and the original is the parent

'''Process Duplication and Unix'''
* Useful operation for concurrency
* Simple interface
* Can be combined with other operations to run programs

'''Shell'''

*These commands make up an important part of the unix shell:
**Fork: The action to duplicate a process
**Execve: Replaces the process running with something else
* Most operations you perform are done by forking the current process and execve to run the commands binary
* If you don't want to `fork` you can write `exec` in front of your command.
*The result is your shell morphing into a command. Once your command ends, your terminal will close

'''Finding documentation from the shell'''
*$ man 'name of command here'

'''Running LS from another program'''

_Execve is a suicide call_
*in C:
**argv and envp are params given to main

execve("bin/ls", argv, evnp);
printf("Hello");

*printf will not run since program morphs into ls

*What if we want hello to print?
**If you just throw `fork()` in you will lose track of which process is the parent and both duplicates will do the same thing

'''Fork: The two faced method'''
* Based on the return value you can identify the parent process:
* Fork returns the PID of the child to the parent
* Fork returns 0 to the child

*In the following scenario, the main process terminates before `ls` runs resulting in the terminal re printing its prompt before `ls` outputs
**C code example:
PID = fork();

if(!PID){
execve("bin/ls",argv,envp);
}
else{
printf("done");
return(0);
}

*In order to avoid this we need to add a `wait()` so that the parent process waits for its child to terminate before it does.
**Cxample C code:

PID = fork();
if(!PID){
execve("bin/ls",argv,envp);
}
else{
wait(NULL)
printf("done");
return(0);
}

'''What happens to a child process when a parent dies?'''
* Another process becomes the parent

*Some process status code examples:
S = Sleeping
Z = Zombie (A process that died but is still there)

*If nobody waits for a process that dies it becomes a zombie
**"If you have a kid, wait for it to die"

''' What is kill?'''

* Clicking X on a window is how we normally invoke the kill command.
** This triggers a signal which is something used for processes to communicate between each other

* `$ man 7 signal` is the command that shows the list of signals

examples:
`kill -STOP *PID*`
`kill -CONT *PID*`

*Some processes can handle signals differently thanks to signal handlers

`$ kill -9` can avoid signal handlers and kill mercilessly

*The `wait()` method responds to `sigchild` which gets the signal of a child dying

'''About `int main` parameters: '''

* char* argv[] is an array of strings
**Each string comes from input separated by whitespace when entering a command

argc:
Count of how many strings are in argv

envp:
evironment variables(state) was passed in

Operating Systems 2018F Lecture 1

2018-09-19T16:22:11Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 5, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec01-20180905.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec01/ here].

== Notes ==
===Introduction===
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
*High performance
*Minimize overhead
*protection between applications
*Isolating processes
*Reliability(runs nonstop)
*Energy efficient
*Secure
*Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>

Operating Systems 2018F Lecture 1

2018-09-12T17:59:18Z

R.samanfar: /* Lecture-2 */

==Video==

Video from the lecture given on September 5, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec01-20180905.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec01/ here].

== Notes ==
===Introduction===
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>

Operating Systems 2018F Lecture 1

2018-09-12T17:59:09Z

R.samanfar: /* Introduction */

==Video==

Video from the lecture given on September 5, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec01-20180905.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec01/ here].

== Notes ==
===Introduction===
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>

=== Lecture-2===
<center>'''Being revised!'''</center>

Operating Systems 2018F Lecture 1

2018-09-12T17:58:44Z

R.samanfar: /* Notes */

==Video==

Video from the lecture given on September 5, 2018 [https://homeostasis.scs.carleton.ca/~soma/os-2018f/lectures/comp3000-2018f-lec01-20180905.m4a is now available].

==Code==

Code and files from the lecture (captured as they were at the end) are available [https://homeostasis.scs.carleton.ca/~soma/os-2018f/code/lec01/ here].

== Notes ==
===Introduction===
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>
----

=== Lecture-2===
<center>'''Being revised!'''</center>

Notetaking for Operating Systems (Fall 2018)

2018-09-12T17:58:03Z

R.samanfar: /* Course Notes */

Notetaking for Operating Systems (Fall 2018)

2018-09-12T08:01:04Z

R.samanfar: /* Lecture-2 */

Extra credit is available for students who contribute lecture notes to the class wiki. You can earn up to an additive 4% bonus to your final grade by contributing notes and improving existing notes.

==How to contribute==

Contact Reza (RezaSamanfar at cmail.carleton.ca) and sign up for a day you'd like to take notes for. Normally only one student can sign up for a day. Sign ups are first come, first serve; however, priority will be given to students who previously have not taken notes.

After the lecture, email your notes to Reza with the subject formatted like "COMP3000-lecture1" (preferably mediawiki formatted). He may request changes from you. Once the notes are in an acceptable format, they will be posted to the class wiki.

==What to contribute==
* Bullet point summaries of topics covered in class
* Code samples from class plus comments or extra explanation
* Time marks to important points in lecture audio/videos
* Links to reliable supplementary information (e.g. page numbers in ''Operating Systems: Three Easy Pieces'')

'''Important''': read up on how Mediawiki formatting works before editing, and use the ''Show Preview'' button before saving your edits.

==How do I get my bonus marks?==

If you make significant contributions that do not need to be edited or improved, you will get roughly 1% per lecture. Large contributions that need some small edits will get you somewhere between 0.5% and 1% bonus. Good smaller edits and contributions will get you around 0.5%. Things like fixing typos and formatting are appreciated but are not worth any extra marks. Bonus marks are granted at Reza's discretion and will show up in CuLearn.

If you're not sure if something is appropriate to be added to the wiki, or you have a question about the bonus marks, ask Reza.
== Course Notes ==
=== Lecture-1 ===
====Introduction====
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>
----

=== Lecture-2===
<center>'''Being revised!'''</center>

Notetaking for Operating Systems (Fall 2018)

2018-09-12T08:00:43Z

R.samanfar: /* Lecture-2 */

Notetaking for Operating Systems (Fall 2018)

2018-09-12T08:00:10Z

R.samanfar: /* Course Notes */

Notetaking for Operating Systems (Fall 2018)

2018-09-12T07:58:43Z

R.samanfar: /* Course Notes */

Extra credit is available for students who contribute lecture notes to the class wiki. You can earn up to an additive 4% bonus to your final grade by contributing notes and improving existing notes.

==How to contribute==

Contact Reza (RezaSamanfar at cmail.carleton.ca) and sign up for a day you'd like to take notes for. Normally only one student can sign up for a day. Sign ups are first come, first serve; however, priority will be given to students who previously have not taken notes.

After the lecture, email your notes to Reza with the subject formatted like "COMP3000-lecture1" (preferably mediawiki formatted). He may request changes from you. Once the notes are in an acceptable format, they will be posted to the class wiki.

==What to contribute==
* Bullet point summaries of topics covered in class
* Code samples from class plus comments or extra explanation
* Time marks to important points in lecture audio/videos
* Links to reliable supplementary information (e.g. page numbers in ''Operating Systems: Three Easy Pieces'')

'''Important''': read up on how Mediawiki formatting works before editing, and use the ''Show Preview'' button before saving your edits.

==How do I get my bonus marks?==

If you make significant contributions that do not need to be edited or improved, you will get roughly 1% per lecture. Large contributions that need some small edits will get you somewhere between 0.5% and 1% bonus. Good smaller edits and contributions will get you around 0.5%. Things like fixing typos and formatting are appreciated but are not worth any extra marks. Bonus marks are granted at Reza's discretion and will show up in CuLearn.

If you're not sure if something is appropriate to be added to the wiki, or you have a question about the bonus marks, ask Reza.
== Course Notes ==

=== Lecture-1 ===

====Introduction====
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1'''</center>
----

=== Lecture-2===

Notetaking for Operating Systems (Fall 2018)

2018-09-12T07:57:36Z

R.samanfar: /* Course Notes */

Notetaking for Operating Systems (Fall 2018)

2018-09-12T07:55:26Z

R.samanfar: /* Course Notes */

Extra credit is available for students who contribute lecture notes to the class wiki. You can earn up to an additive 4% bonus to your final grade by contributing notes and improving existing notes.

==How to contribute==

Contact Reza (RezaSamanfar at cmail.carleton.ca) and sign up for a day you'd like to take notes for. Normally only one student can sign up for a day. Sign ups are first come, first serve; however, priority will be given to students who previously have not taken notes.

After the lecture, email your notes to Reza with the subject formatted like "COMP3000-lecture1" (preferably mediawiki formatted). He may request changes from you. Once the notes are in an acceptable format, they will be posted to the class wiki.

==What to contribute==
* Bullet point summaries of topics covered in class
* Code samples from class plus comments or extra explanation
* Time marks to important points in lecture audio/videos
* Links to reliable supplementary information (e.g. page numbers in ''Operating Systems: Three Easy Pieces'')

'''Important''': read up on how Mediawiki formatting works before editing, and use the ''Show Preview'' button before saving your edits.

==How do I get my bonus marks?==

If you make significant contributions that do not need to be edited or improved, you will get roughly 1% per lecture. Large contributions that need some small edits will get you somewhere between 0.5% and 1% bonus. Good smaller edits and contributions will get you around 0.5%. Things like fixing typos and formatting are appreciated but are not worth any extra marks. Bonus marks are granted at Reza's discretion and will show up in CuLearn.

If you're not sure if something is appropriate to be added to the wiki, or you have a question about the bonus marks, ask Reza.
== Course Notes ==
<center>=== Lecture-1 ===</center>
====Introduction====
'''why is an operating system important?'''
*Only certain programs run on certain operating systems
*Customization
*Interface
We program at different levels of abstraction
*Abstraction - make system easy to use, divide program into smaller understandable
pieces
*most portable - the web
**works on almost all operating systems
*the web browser is an operating system (eg: chromebook)
'''How do programs run at the same time?'''
*Originally, one program used to control the whole system, now for example there are
around 270 threads running at once on a laptop
*A program executes instructions → '''the processor fetches an instruction from
memory, decodes it and executes it'''
*Similar to a political system: Authoritarian dictatorship
*Virtualizing the CPU: illusion that a single CPU has a large number of smaller virtual
CPUs.
*Policy: to determine which program runs first
'''Operating systems'''
*body of software to determine which programs to run, allow programs to share memory,
enabling programs to interact with devices
*the OS takes a physical resource (such as the processor, or memory, or a disk) and
transforms it into a more general, powerful, and easy-to-use virtual form of itself
*AKA virtual machine
*provides some interfaces (APIs) that you can call (system calls that are available to
applications to run programs, access memory and devices, and other related actions,
aka standard library)
*Resource manager - give programs resources based on priorities
'''Goals:'''
**High performance
**Minimize overhead
**protection between applications
**Isolating processes
**Reliability(runs nonstop)
**Energy efficient
**Secure
**Mobility
Unix
*Has variants - linux, minux
*Programmed in C
Windows
*VMS
*Microsoft bought a team to build an operating system
====Hello World - Sample Program====
C programs start with '''main function'''
[[File:Lec1-1.png|none|800px]]

Compile and Run:
[[File:Lec1-2.png|none|800px]]
Two different ways to compile:
*One does more work and it is smaller
*One does less work and it is larger
[[File:lec1-3.png|none|800px]]

'''How many lines of code are running in a system?'''
* 100 million lines of code!
'''Questions to ask:'''
#Who calls main?
#Where do the arguments come from?
#Who's taking the return value?
#Who defines headers? Where do they come from?

'''Mental Model'''
*Don’t need to know exactly what happens
*Know the strategy to figure it out
*Know when to ask the questions
*Know when it matters
*OS → the code that runs under the application
'''Who is in charge?'''
*the '''first program''' that runs on the system is in charge
**may delegate its powers to others
*make sure it only obeys your commands (self preservation)
**eg: if you get a command from someone else, ask me first
**add a level of abstraction: authorization
**downside: labour intensive (time consuming to approve every command)
*create rules to determine when authorization is needed
**when does the OS need to get involved
**this is costly, so leave instructions behind so that is can make decisions
**'''take charge of dangerous operations'''
'''Cpu modes:'''
*'''Supervisor mode''': access to all hardware resources
*'''User mode''': access to limited resources, as defined by code running in supervisor mode
'''Definitions:'''
*Kernel: code that runs in supervisor mode (If your code runs in supervisor mode, you are
in the kernel.)
*Process: “running program” (Processes run in user mode)
*System call: process making a request of the kernel (costly)
*portable: able to work on different operating systems, a program that can run in many
different environments.
'''APIs'''
*an application programming interface (API) is a set of subroutine definitions,
communication protocols, and tools for building software

'''Memory'''
*read - address to access data stored there
*write - address and data to be stored there
*virtualizing memory - each process accesses its own private virtual address space
**The OS maps this memory to the physical memory on the machine
'''Concurrency'''
*working on many things at once in the same program
*multi-threaded programs
**thread: a function running within the same memory space as other functions
*race conditions
'''Presistence'''
*how do we not lose data
*volatile - when the system crashes, the data in this memory is lost (DRAM)
*hard drive - long term store
*solid state drive - alternative for hard drive
*file system - manages disk storage
*handling system crashes
**journaling or copy-on-write
<center>'''END OF LECTURE 1</center>
----

=== Lecture-2===

Notetaking for Operating Systems (Fall 2018)

2018-09-12T07:54:56Z

R.samanfar: /* Lecture-1 */

File:Lec1-3.png

2018-09-12T07:20:06Z

R.samanfar:

File:Lec1-2.png

2018-09-12T07:19:55Z

R.samanfar:

File:Lec1-1.png

2018-09-12T07:19:33Z

R.samanfar:

Notetaking for Operating Systems (Fall 2018)

2018-09-12T01:14:53Z

R.samanfar:

Notetaking for Operating Systems (Fall 2018)

2018-09-12T01:14:18Z

R.samanfar:

Notetaking for Operating Systems (Fall 2018)

2018-09-10T00:24:28Z

R.samanfar: /* How to contribute */