Operating Systems 2019F Lecture 17

Video

Video from the lecture given on November 8, 2019 is now available.

Notes

Lecture 17
----------

special files: files that aren't directories or regular files

Three kinds of special files:
* character devices
* block devices
* named pipes/FIFOs

first letter of permission block:
 -   regular file  (hard links)
 d   directory
 l   symbolic link

 b   block device
 c   character device      <--- these three are "special files"
 p   named pipe/FIFO

A special file really isn't a file; instead, it is an interface that
uses a file-like API
  - you use standard file system calls to do weird things

Note that special files way predate things like /proc

Why do we have them?
 * block devices represent mass storage (disks)
 * character devices represent most other I/O devices
 * pipes are inter-process communication (IPC)


Why do we need block devices?
 - manipulate mass storage, e.g.
   - format disks (lay down filesystem on storage)
   - repair/recover filesystems on disk
   - low-level backup

Normally you shouldn't be playing with block devices!
 - because you interact with them indirectly via mounted filesystems

The kernel implementation of block devices is a bit complex
 - lots of machinery for caching

If you wanted to learn about how block devices work, look at the implementation of loopback devices, /dev/loopX

Why "loopback"
 - normally you have this situation

  filesystem    <-- /home/soma/foo
  mass storage  <-- /dev/sda

So here foo is stored on /dev/sda

-- but with a loopback filesystem

  filesystem   <-- /mnt/somefile
  loopback     <-- /dev/loop0
  filesystem   <-- /home/soma/bigfile
  mass storage <-- /dev/sda

/home/soma/bigfile is mounted on /mnt
storage for bigfile (and hence somefile) is actually on /dev/sda


Loopback devices are very useful for
 - learning about filesystems
 - making ISOs (CD/DVDs)
 - accessing virtual disks of virtual machines

Note the loopback device just allows us to reuse the filesystem implementations already in the kernel.  You could always pull them out and treat the file as one with a really baroque storage format


Character devices
 - everything but mass storage
 - tapes, mouse, keyboard, ttys


named pipes/FIFOs
 - examples of the producer/consumer problem
 - need to pause producer when consumer gets behind
 - need to pause consumer when producer gets behind
 - use buffering to allow each to run at their own pace (when not paused)

General pattern
 - producer runs until it fills a buffer.  When buffer is full, pause producer
 - consumer runs until buffer is empty.  When buffer is empty, pause consumer
 - must resume producer when buffer is no longer full
 - must resume consumer when buffer is no longer empty


What is the mechanism for pausing and resuming with pipes on UNIX?
 - put the process to sleep when it reads or writes
 - producers write, consumers read

How do we make pipes?
 - at command line, use |
 - in a program, use pipe() or pipe2() and generally dup2()
   - consumer gets first fd, puts it on stdin (fd 0)
   - producer gets second fd, puts it on stdout (fd 1)  <--- said stdin in lecture, that is wrong
   - note the producer is the one to the left of |, consumer is to the right
 - mkfifo or mknod p
   - allows for "persistent" pipes that are used by multiple processes
     at different times, a way to do rendezvous

eBPF versus modules
 - both let you inject code into a Linux kernel
 - eBPF is safe and restricted
 - modules are not safe and are not restricted