Operating Systems 2014F Lecture 1

Audio from the lecture given on September 5, 2014 is now available.

Notes

Operating System: What is an Operating System? A layer between hardware and your applications. Operating Systems are software, not hardware. The distinction between these two is not always so clear. This diagram is misleading. Bits of the operating system may also be in the application layer. (For example, libraries) Basic conceptual model of what is happening inside the computer is that the operating system code is both:

• separate from your programs,
• and integrated in with your programs.

The reason for this is because operating systems have a very distinct role. Which is to make your life easier. Not your life as a user, you as a programmer. That's the basic goal of an operating system. It takes the computer you have and outputs the computer you want. (This is not an easy task, the hardware guys have made a mess - hardware is complete chaos.) Most of the time you don't want to do that, the hardware from one machine to another will vary.

Real Hardware

• variable, quirks (bugs, problems, faults, mess-ups / didn't do things right, instead they do things differently) - abstraction
• most hardware expects a single master - resource management / sharing

* most hardware expects to be told what to do by one program, and only one program will actually give it instructions 
* They need to be consistent, not contradict each other.

On a regular computer we want:

• portability -
• uniformity
• run many programs concurrently

* we don't want to run just one program at a time. 
* Look at modern computers - they have gigabytes of RAM, they have multiple cores, they can do so many things. 
* So why do they have to just run one program at a time?

We want to be able to run on multiple systems, and want them to operate / behave in the same way. You can make portable code by putting in a lot of conditions to deal with lots of things. But we want to be consistent as much as possible.

Real Hardware VS. What We Want our Hardware to be capable of (Conflict / Gap)

The Operating System is what bridges these two sides. Gives you something you live on a regular computer where real hardware acts like this. They do things you can't normally do. The trick with Operating Systems is that they do things you can't normally do, in that they have powers that other programs don't have. They get around the quirks and variability of the hardware. We get abstraction. Operating systems take this idea pretty far. Translates the computer you have, and outputs the computer you want to program.

Operating Systems enable sharing between programs. If you think about a computer as not running one program, but running a bunch of programs all together, think of them like living entities. Now you have some sort of social structure, some model of governance.

What style of government does an operating system impose? An absolute dictatorship - highly authoritarian. Command and control autonomy in the strictest sense. The OS is boss! () You have to follow the rules (as a process that is executing) You step out of line you die. That seems kinds of mean. There is good news and bad news from that. I'm talking about single computers, you put a bunch of computers together, you have another form of government. It's called anarchy. It's called THE INTERNET. That's why you have 4chan, etc. Distributed operating systems - going beyond anarchy...

If you look at the history of computers, what we called operating systems early on. They all provided some degree of abstraction / resource management. They tended to provide much less. How many have used MS-DOS? The thing that used to underlie Windows XP. It doesn't allow you to run multiple programs, at least not very easily. It lets you run multiple programs if you kill one, and run the next one. That's how all personal computers used to work. Why didn't they run multiple programs? Ans: Because they were pathetically underpowered. 1 meg of RAM dealt with 8 bit quantities at a time. You don't want another program running sucking up your resources that your other program needed to run. Fewer and fewer of them don't have an operating system. As you get more and more resources in a computer, you want to do more with it. All operating systems are strictly unnecessary. They slow down systems. But they make them much more usable.

In the research literature: Operating Systems are too expensive, we need to do this radical change in order to make them less expensive. Operating systems over time take more and more resources. Because we want them to do more things. It does result in systems of remarkable size. It used to be programs of 50,000 lines of code were considered kind of large. How much code is in a modern operating system? A modern web browser is 5 million lines. the Linux kernel is 15 million lines. Windows 50 - 100 million lines of code. War and peace is 100,000 words. These are big, very very big programs. So big they are not humanly comprehensible.

Abstraction means you don't have to understand how it all works. Older operating systems courses like to have implement an operating system from scratch, or be able to see all the code you are running. How do you deal with systems that are ridiculously complicated. How do you get them to do what you want? How do you debug the code you have added? How have other people chosen to organize it? This course is how you deal with the mess of modern programming. Instead of playing with an entire small operating system, I want you to work with and make changes to an operating system that you all use. Even if you are running windows on your desktop / windows phone, it's still running behind most of the web services that you access. If you grew up with windows - it is now on the minority of computers in the world. Most computers are embedded systems, or servers. Android runs Linux. IOS runs a variant of MAC OS - mostly FreeBSD - mostly Linux. Windows has it's own way of doing things, but it's not that different from Linux. Linux isn't bad, most of it's warts can and are being fixed. You aren't going to see another operating system take over. What's an operating system for except to run code? We have lots of code we want to run. You want to run an operating system, and then you make a compatibility layer - you basically have re-implemented another operating system.

What is this thing we are abstracting? Let's talk about real hardware. Von Newmann architecture - pioneer / history of computer science (Turing's Cathedral - Book) Talk about CPU, RAM,

Microprocessors, store data in ram, processes in ram, store results there, also does I/O (disk, keyboard, video screen) Notice there is no notion of programs. There is just code. There is just memory. Operating system is really about putting abstractions in between the components. Talking about abstraction within memory, we are talking about address space - a numbered sequence of values. Addresses go from 0 to some big number (2^32). A program could exist somewhere in there. Store instructions here in binary language. Talking at a high level here. How mysterious does this seem, the notion of machine code? Take some C code, transform it into instructions, to make this more concrete. Trying to program your brains by giving you abstractions. The RAM is an address space. The CPU contains registers. These are particular bits of storage attached with logic to transform things. What is I/O? Disks, video, keyboard. What an Operating system does is it changes these things into abstractions. Things like Files, network connections, web, TCP/IP and stuff. They also turn these things into processes.

Process - key abstraction behind an operating system, placeholder is a running program. Operating systems are running processes - they are logically running at the same time because if you have 100 processes on a system, you only have 4 cores, most of the processes aren't running at any given time. They are shared. How do I get them all running at the same time. Well you multiplex them. Say you have 10 assignments to do, you have 2 weeks to do them. You get 10 assignments at the same time, you divide up your time. You might only want to do part of each assignment, to ensure you do part of all of the assignments, instead of not doing anything on others. You have to figure out some way to allocate your time to do the different assignments, but you are only going to do one of those at any given time. That's what the computer does. It just does the switching really fast, so it seems like they are all running at the same time. That's the key way you share the CPU.

To understand how operating systems have changed, you need to understand Moore's Law.

Moore's Law:

• The # of transistors on a chip (of fixed size) doubles every 18 months.

Things are slowing down a bit. Intel is staying on this curve. Other companies are falling behind, because it is becoming really expensive to stay on it. It is an exponential curve. They never go on forever, they cannot. Wikipedia of Moore's Law This is why you can fit a billion transistor's in your pocket. CPU, RAM and other devices are driven by transistors. One underlying theme to keep in the back of your mind, realize the abstractions that we are using today, were all developed in the late 60's and early 70's. The systems that those things were created for had MB of RAM. Now you are carrying around devices with GBs of RAM in your pocket. Abstractions of operating systems at a fundamental level are trying to deal with how to do these things, but the things underneath become bigger, the abstractions aren't so useful. If you have code that used the old abstractions you can't just throw them away.

How many files are on your computer? Talking like 100,000 files is no big deal. That's a lot of files. A filesystem is a way of abstracting hard disks, a hierarchical storage for naming things. How useful is that abstraction for finding things with hundreds of millions of files. Not that useful, it's too big. Can you get rid of it, no because you still use it to do things. Weird thing about IOS - there is no shared filesystem on IOS. They divided up the applications to remove shared filesystem. A usability issue - users would lose the files. What is a camera roll? it's files, they give you your contacts - that's a database stored in multiple files on your filesystem. The point I'm trying to make, because of Moore's law a lot of the abstraction that you are going to learn in the course are kind of quaint. They are not going away. They are the plumbing.

The classic theme of this is something called:

Virtual Machines

Here the VM I'm talking about is using Virtualbox. This is used to talk about virtualizing resources. (for example: Virtualbox). You are going to be running an operating system as a program. If you think about this for a minute. This is crazy. Instead of running one operating system and having each have their own applications. Instead you have one operating system per user. I'm running an extra 50 million lines of code per user. (Example: AMAZON Cloud - Hypervisor - on top of that you are going to run your own instance of the operating system.) Unix was designed as a multi-user operating system. Designed to allow many people to share the same computer. But instead of doing that now we run entire operating systems for one user. This is horribly wasteful. But this is also very useful. It's useful because we can now run different operating systems for different users. Moores' law allows us to be sloppy and lazy. My program only runs well on this particular operating system. (Example: Ubuntu 12.04, well I'll keep a vm around that just runs Ubuntu 12.04) Today we have resources to burn. How do you take old abstractions and make them work today?

Solid state disks are they a disk? Do they make round chips? No they are chips, they are storage, it is silicon. We understand it that way because the abstraction we use to access it is that disk abstraction. Sequential reads are much faster than random reads. Not true for SSDs. Random reads are the same or faster than sequential reads. Writes are very expensive, you can't write too many times to the same sector. It is dead. You have to spread out your writes using wear leveling. Wait a second, filesystems were made to abstract the writes. You put an operating system on a solid state disk. You are actually running a filesystem on top of a filesystem. We layer this, because if you put your own filesystem on top of it, you will blow it up. Flash drives have operating systems. It's turtles all the way down, see :

Hints for Assignment 1 Question 8:

Why is the browser like an operating system?

• Article 1
• Article 2
• Article 3

For the scriptable application section.. Think about what happens when you save a word document, and you have added an image, or a script within the file.

• Research Paper