We can get started
So up those i'll talk about that figure in a little bit
So advice on Co. so let's say where are we right now we are.
We have this week's lecture and we have alright. We have this week's lecture, and 2 more next week.
The assignment is due in a week and a week i'll go over solutions to assignment for
I I last class. If you go back last lecture, I went through a lot of things for showing how to how to put multiple strings into a buffer and how to return multiple items from a function.
If you have other specific questions i'm happy to discuss let me know
But what I want to talk about today was
So when we run a program like top, you see
Okay, I'll, i'll answer that question real quick so The question is, How do I?
How do you find the 10 parent process? ids it's a tree? You have to walk up the tree, you have your C. So if you look up, how get ppid, get the pair of process?
Id works. You get the code for that. that gets you the parent process, and then from the parent process, the parent task.
You then look for its parent. Then you keep going until you get to the top of the tree, which is when you get to a knit.
So that's what you have to it's a recursive process.
First figure out how to do. Get ppid and then you walk it, you'll need sn print off.
Yes, you, because as well your technical idiot you could do other ways.
But sn printer's probably the easiest way to format things in order to put it into in order to put it into the buffer.
If you've been gang seg faults as output, that means your
If you're getting seg faults in your output that means that you're not you're accessing memory.
You shouldn't so be very careful about your pointers and what you know.
Memory. you're access is probably something off there you can't call get ppid the system call.
But you can look how the get ppid system calls implemented, and you can run that code that's the trick you don't call the system call you call you just literally copy and paste the 3 lines of code that is get
ppid into your function in order to get the parent and you stick in a loop, and you hit it right for up to 10.
But if you're getting seg faults it means that your you have to think about how your alky memory. Now, I specifically gave you lent your room to do things in, because I've allocated for you a
page, which is 4 K. So you have lots of room for building up your string.
There's no reason for you to do more than that you shouldn't have to allocate more memory for that
You. we'll, have to allocate room for an array for passing, you know, addresses between the get get physical info and stuff, but that's just just need an hour of 10 you know things that's 10 times
8 is 80 Byte that's fine to do on the stack.
But do that allocation as a local variable in the caller, not the calling
Hey? Other questions about the assignment
Okay, i'm not talking I last call i'm not talking more about the assignment until the end of class, unless it comes up in contact
Moving on. So what we? So what was I software? So here we go running top.
This is top of my personal machine here, where i'm running zoom. So of course it sucks up lots of resources.
But the thing I want to talk about is this virtual resident and shared.
Now, what this is is this: is this virtual: The size of the virtual address.
Space resident is how much of physical memory is taken up and shared is how much is being shared.
Yeah, what I want. this is where I wanted to talk about.
I made this little diagram. So so here this is kind of a standard view of hardware, kernel, and process.
So you say you have your processes, emacs basketball or firefox.
I think they're running in processes so these little lines here are meant to designate their in separate address spaces, living in their own world when they want to get out of that world.
They have to interact with the kernel. They make system calls system calls.
Get them into Colonel Land and so that you know that's the interface between these 2, and then you know it returns information.
But of course, the kernel. how it does this job!
A colonel can see all of all the processes interact with all of them.
But it also has to interact with the hardware All when I say hardware.
Now we say the kernels you know really that's you know the Cpu and memory are kind of like directly.
There we talk. Hardware really is more I O. and and when and the hard way is dealing with, as in, you know, screen keyboard and stuff in in interact with those Via, interrupts.
And Dma, but he is an interrupt is analogous to a signal, but it's basically a message from the hardware to the kernel.
Say, pay attention to me. i've got something for you and Then you can go, and, you know, get the information or give it information.
Dma stands for direct memory App: So so this is kind of, you know.
High level view of these things. but in this sort of picture this does not account for a notion of shared addresses.
So let's try thinking about i'm gonna we'll see how well this works.
So
So let's say i'm going to you know we have a process here unless you know we have our process.
Yeah, we'll call this backs so this is our address space for emacs.
And then I have you know we have another one here we'll say this is bash.
So both emacs and bash. Well, just
Emacs and bash both have inside Here they have in their address.
Space is a live c. because they both link in the C library
We need a binding to switch between artist mode and not.
Hmm! It probably is a keystroke for that.
But, anyway, is it possible to draw arrows as well?
I can definitely draw lines. let's see let's see if I can draw an arrow here.
Into a line
Okay.
So this is artist mode in emacs That's where I get. So you can see i've got these. Can you see this little popup menu when I do this the trying pen line all that stuff yeah you can see
it
Set fill set line. Oh, I can change the line.
I lost the character. Okay, this set fil name changes to color.
Let's see, I I think Yeah, you can set the you can set the character fulfilling, so like if I do this.
Set fill i can do it to let's say Dot. So if I do the fill to dot, and then if I do draw a rectangle, it should yeah put stops in there
Seems a lot better than Nano I don't think Nano has this, anyway.
And we'll be i'm going to click with I do not want my rectangles to have dots right now, so let me.
So I actually have to middle click, to get this menu so i'm not sure touch pad you probably just double taper.
But again this is Emac. You can rebind everything.
You can make it work. How you want I wanted to erase So we're going to erase this we or else I needed pictures for today.
So it was like, Oh, yeah, artist mode. So anyway.
So i've got emacs i've got bash here
Let's see, now, is there this is all the things switch out of artist mode.
Okay.
Fine. Never mind up.
So we say we have 2 processes and they're both running bash and emacs
So we could just load these separately into each process, lips, c.
But that would be very wasteful. So what happens instead
So what i'm saying is that you only load one copy into
There's only one copy of lip c and physical memory but it's shared between each of these things.
So
Is this the Jersey stuff? So is this. So this is
If you want to do the sharing, it has to be doing dynamic linking, it can't just be doing
It has to do dynamic linking it can't just be doing exactly. now.
You might notice when you if you look at the physical pages for things like put s between executions, they should stay the same
So Jason just saying is this: why code is stored in the text here, right right?
Well, the variables are on top, so where things are in virtual memory is actually completely a convention.
There's some practical things about like you want to make sure we'll sort of run into each other. but it's just It's just convention there, I mean it could be anywhere, as long as they you know you know running
into each other, and especially the 60 four-bit address space.
You have lots of room to put things where you want the main thing, though, is what's close together has to stay close together, you know, in terms of like Lib. c.
Is all going to be mapped into one portion of the address space, because it needs to be a contiguous range of addresses.
Otherwise it you know. let's see what nowhere with the rest of it was
So we have this
So you know what system call is is the thing that we do in order to
In order to load this into memory in a way that will be shared.
It's a system call you've all seen
Yep
If you think about this, you're the page table we talked about, is this very wide tree.
So, depending on the circumstances how it is, it may actually want to copy parts of the page, table and stuff, but it only has to load the file into ram once and that file. it.
Then says, Okay, this file isn't this part of physical memory it has a bunch of you know, a little subtree of the page tail to correspond to that portion.
And then other page table can just point to that, or it can duplicate.
If it has to but that's that's what's logically happening.
Now that means that. So
So when you're talking about mapping more specifically process, address space, you map it into the physical address.
So when I say process, address, space.
But when we
So post process need the same file we couldn't lock on a locks it's faster like what so in both processes want.
If 2 processes want to read the same file, if you just map it in no problem.
If they both want to modify it. you should ensure some forum mutual exclusion, because you can't guarantee they're actually running to the same thing.
So that's because just like anything you have to enforce mutual, exclusive page tables are just after abstraction.
Real date is just a byte stream page tables are an abstraction over memory, but it's not a byte stream.
It's it's ram so I say it's not a stream.
I think it was a continuous you know what a byte streams like you're reading from a file.
You can bite after byte. here. This is addressable memory, random access.
You can access each byte in memory specifying its address That's what you know that's what you have with it.
So it's not so I won't call it a stream but it is a a set of sequential bytes.
It's more i'd say think of it as an array of bites.
So good? question.
So one reason why they could be nothing be there is because it was allocated, but not used.
If you don't actually access the page that means that Well, the Crookins go. Oh, you asked for a gigabyte of ram.
But you're not using it. so Why, do I have to devote all that you know those physical pages to it you aren't.
You don't need it when you access it i'll give it to you.
But until then we'll just forget about it
So it's also organizing perspective
Yeah. So is that fiscal as low as organized into our dressable pages.
It's a page table as a way to figure out which rams.
They can pay just right. So I mean which ram stick that's actually the physical address, the physical address is what maps to the specific stick of ram in the chip, and everything when we talk about you know all those that particular things the
implementation of ram gets abstracted away at a low level by the hardware into physical addresses.
Virtual addresses as a mapping is something that there's hardware acceleration for it.
But it's finally managing by the operating system and it says, Oh, you want to access this virtual address.
Well, let's look up with a physical addresses and then the actual memory.
Access happens on the corresponding physical address
Yeah, the the motherboard, when you talk about interleaved ram and things like that.
A dual channel multi channel ram what they're saying is, is, when you access certain bytes they aren't actually stored like this address range.
Is it on one chip in this other chip you actually are interleaved, so that when you are accessing you're actually accessing parallel like this bite will come from here the next bite will come from here that
is. Probably they'll be read in parallel and how exactly that interleaving is done depends on the hardware. It's typically not on a bike by byte level.
It's typically on larger chunks the bottom but that's all hardware design at from our perspective.
We don't really care. We just but the abstraction we see at the operating system level is physical page.
You know physical memory. Yes, but again, physical memory is such a chaotic mask.
Even the kernel doesn't operate in physical memory it has its own virtual address.
Space for its own code only, you know. and then everything is going through the memory management unit to decide where it is. because, remember, physical memory is just pages everywhere.
This whole chaos of things
So like if we say so, let's say you know if if we're looking at pages, you know
B cm d
Here.
Exactly. Yeah. Yeah. pages with the real data stored page tail stores. It's: yeah page tables are mapping
So let's go back to the
So what are we talking about here again with this is virtual address.
Is, how big is the virtual address space that is allocated for that process.
Resident is how Oh, I I I don't I forgot something
So okay, so parts of it. So what is it? allocate?
But not been used. that's one reason why the address space might have been
So the other aspect of so so. the kernel doesn't have to give processes all the memory that it asked for.
Yeah.
They're always exceptions that sometimes isn't always it's not you do some prefetching and other things.
That's it's more cost but fundamentally it's lazy, and so
So this whole concept, really called, demand paging stuff is loaded from disc only as it's needed
Hmm.
Hey? So So when you say pages act like Malik and free George, i'm not quite what you mean that pages act like Malik and free, I mean certainly underneath Malcolm free.
There are these pages.
So so how does memory so? how does? Only how does memory that has only a virtual and abstract allocation, plus a physical look on the patient was just missing part of the address so
So
So page table entries.
One of my cats is downstairs, and I can hear them just yowling away.
He's just going. No, No, I don't think you can hear I wanna close my door.
We want. Still, he can come upstairs and say, Hi!
That's tab here. i've got shift and Roshi are both on the floor here.
I'll show you guys later. yeah yeah he wants to come to you.
Yeah. So So what metadata do you have?
Because, remember, we
So if we're talking about a pointer here in a page table entry.
Yes, you have 64 bits most that you you need for potentially specifying address, but you have at least the offset worth for specifying meditation.
You have at least 12 bits Typically, most all those are not used, but they are used for.
Several things. so one, you know.
Just like files. Each page can have a read right and execute bit.
So is this a readable rival? X. Few little bit
And then also we can have things of like, Is it
We are present, you know, is is the page present? Is it a valid page
So
So there can be more. but at least you're gonna have these kind of bits on a modern system.
You're gonna basically have Oh, oh, yeah and and
So there's an access bit in the dirty bit which has anyone written to this page and has anyone read from this page, and and then you have the present valid bit, and then you rewrite execute bits so
So if it's not present when you see the page is not present, it has to check some other data structure, and go like, Oh, it's not present.
Should it be present? Is it actually there? So oh, this is actually a mapped in file.
If this is a mapped in file. Oh, I haven't actually loaded that part of the file from Disc. I better go load that from disc, have it in memory.
And now, and i'll put the process to sleep until that data has been loaded, and then once it is then i'll go.
Hey, here's your data
But a choice. I mean it has a choice ask to decide what it's going to do for the kernel there.
Yeah, memories allocate the entire pages needed. Yes.
So when you first use a page, will amount the entire thing into physical memory.
Yes, exactly.
It's very easy for the kernel to over commit memory You can have given too much out, because
So you know what happens so so there's this constant pressure
So let's look back here. This is telling us stats about how much memory is available, and this is very important to understand is saying, like, we say, swap.
So my computer is actually got swap is I? I have a file on disk.
So yeah. So
So I have a swap file which is a 2 gig swap, file, and if you look up here you'll see it says, swap!
I have also on top swap the exact same as the page file.
Yep. the reason the different their routine swap in paging is swapped as an older term.
So when you would have out of oh, when you'd run out of ram, it would take an entire.
I all of a running program, all of a process. Put it on disk and bring another one into memory.
You would swap them nowadays We don't move processes out and and entirely we do it portion based on the pages that our need, some pages are tempted off to just some page so loaded in so a
page file is a more accurate term, but the older terminology is walk, file a a swap.
Still persists in unix world. but you Look at this diagram here.
What's it saying is that I have 2 gigs of swap most of it's free, but I've actually used half a gig, half a giggle is tightly used but then, also if you look at
my ram. So this so this so this! So we have this.
I have almost 16 gigs on this I have a 16 gig machine, and it's saying about 14 gigs of it are available.
Of that 2 gigs of it are free. 5 gigs of it are used, and then 6 gigs of it are buffer cash.
So I think this 8 gig so what it says available memory. This available memory, I think, is i'm not quite sure exactly what that term is. So I think that's roughly like buffer Yeah, Probably buffer
cash plus free you know what's free minus
I think minus this. let's use have to check that exactly what that's calculated.
But the the important thing to note here is
It's called when wrong. This is called vagrant U. W.
E. Thank you
So free memories amount of ram available, you know pages that are ready to be allocated.
So
So we have. We have this big pool of ram.
What do we use this round for
Don't do that, i'll do that here you go she's jumping on a lamp.
She was sitting nicely in a corner of maintaining stuff, but then she said, Jump on the way up my corner.
She's getting bored. so i'm about to kick her out, but she can say, Hi, right now, are you saying hi to everyone.
Okay, comma, How are you going
Yeah, seriously.
She is. She is not too chunky tab is chunky but she's not too bad.
I I just had a weird position there. what was I saying?
So the we using great So Ram is being used both for running programs and for
So why do we store copies of files in memory?
Because it makes file access much faster, because ram is generally much faster than disk.
Even Ssds ram is normally faster than discs.
So you. So the process is access. Reading from a file. The file has to be gets often loaded into ram and and modified.
There.
So what happens if you right to a file but you're actually writing to a copy of the file that is in Ram.
And then you computer loses power. so there's multiple levels of cache
But and cache is used to speed up access to right.
What on modern systems cash is not directly programmable.
You can see it side effects, but it's not part of the programming model.
What you're part of the programming model is just ram so, and ram is a pro is appears to us to have uniform speed access.
Of course, in practice there's the cache is working to make apple access to certain parts of memory and certain access faster.
But we don't see that programmatically Now There have been architectures where you could directly access the cache.
Hey? one! No game console where that was the case.
No, I not the xbox. the Ps. 3. Yeah.
The Ps. 3 with the cell architecture. If you look at at the way the cell processors were created, they basically were giving programmatic access to cash separate from main memory.
And one side effect of that is the ps 3 was a real problem to program.
And so essentially everyone's given up on that and you you don't have this separate thing for program will cache
It's been tried. It makes programs unhappy so we don't do that exactly.
Yeah, it it was real good.
So so. But I just want. But first this point here, what happens to a file for store, and then the commute is power
Can you just pin something in cash of a dev wanted to?
There are some ways of telling the cpu that certain things are important, and should stay in cash and up, but it's all advisory.
You don't have I mean this ultimately the cpu controls.
What happens this you can have given hints but you can't you'll directly control it.
I mean again. This is this is not that it happened to be this way.
It's that we do it this way because if you don't do it this way.
It's hard to make code portable and performance but so there!
So there's this problem here that you know if you write to a file.
It's stored in ram and the computer lose power you moves the changes to the file.
So you have to write them back to discretely
So in the background, like constantly you're keeping the disc running to write things out.
But it has to be kept up to date. So the changes are word files in ram, and the save button is useful.
So when you talk about auto saving auto saving is an application level feature that the that thought process itself is riding portions of memory.
So
Yes, so normally, when you write a when you write to disc, it's considered properly a problem
So when you so so word will say you know if when you're editing a file, it's just in ram.
It's just in the processes address space so if the the computer terminates.
Of course, all that's gone word will write things to disc periodic. That's the auto save where it just takes the contents of the file and writes it to disk quickly that should be persistent except that actually when it did
that right? It didn't go to disk directly it actually goes cache of disc that we have in ram.
But then the operating system goes like, Oh, yeah, this was written.
I better write it to disk, and it does so jelly in a pretty judicious fashion.
But it doesn't happen immediately, particularly if the system is under any kind of stress, like if you just saw a lightly loaded system, it's gonna write it you know, basically, immediately.
But if it's a more heavily loaded system there's lots of things happening there may be a bit of a delay before it gets to disk.
So
I know auto save depends on it. Because remember this: what when I just hit save on this file here.
Technically speaking, it didn't necessarily get written to disk immediately. it got written, it got it got logically written to disc, but it actually being committed to the physical medium of the storage whether it's a spinning
driver and Ssd. happens a bit later, how long that can take can vary, because again it's doing all this operations file operations first happen in ram, and then later our committed to disc
Yeah. so so honestly, I would say most of the the fast saving mechanism of my Microsoft word is
Oh, one. By the way, one thing about fast saving on Microsoft word is the auto, save only works.
If you name the file. I have known people who have lost lots of things because they work in word, they didn't actually name the file, and so it never got auto safe to disk.
I find that one of the stupid signed decisions.
This is exactly what i'm describing is buffered I o this!
This is buffered Io. This is what I mean by buffered Io, using ram as a buffer
It will. It will not auto save with a default file name.
You have to give it an explicit file name.
So Ram Controller flesh how ram I I was saying, like the When I say this battery backed ram, I mean it's acting like a disc.
Is that, like a very fast disc that's what it is yeah R. R. R. R. R.
Ram ne needs power to work so that's why if it's battery back ram.
It'll it can be persistent
I kind of want to talk about file systems here. Well, I think that's
We talk about file systems here. A laptop is a battery back computer.
Yes, it is so. laptops can be. but but if you do, a hard reset of a laptop like you would just hold down the power button on a laptop that will cause it to lose power and any
unsafe files can get lost. so
So when the Os starts running out of memory so like if I yeah, if we're looking here and this if this the amount free here starts getting too low.
Now the buffer here is very large this is like 6 6 and a half gigs of of Ram is, you know, pop up. That's good, because it's just saying i'm gonna use this so everything is fast, I might as well
keep copies of things off disc and memory. Because why not?
But as memory gets low, it's gonna have to start making decisions
So, even though like when you move, how Lru was applied to boxes.
Have you ever heard this, that when you move to a new place any boxes, you haven't opened after like 6 months or year, you might as well just throw them away, because yeah, didn't need them in the last year, so you're probably not going
to need them, the future. So just chuck them that's basically the same thing here
Yeah.
Is there a range of for so it's not about having expensive computer, whether you should use have some sort of uninturpable power supply.
The reason is, if you have things that you care about in terms of getting corrupted.
Modern operating systems are actually really good at keeping things generally same. There's voice techniques I haven't gotten to yet.
But
There's there's a lot of heuristics for deciding when memories low and stuff that's it's partly it's not just based on how much is there but how quickly is memory being used there's
a whole art to deciding exactly.
Ohx cables. Yeah, So yeah, so that that's what we're talking with Yeah.
I am looking aside, because can I show this I Don't know if you can see this, do you see a little?
Can you see it? But yeah, you can see that little paw there coming out at 6.
Shift is really being okay, She she got some
Oh God!
She found this, which is some thread that sure that she was playing with, and she had stuck it under a door.
We probably don't want our playing with this I will have to have a talk with do not keep roles of string out for the cap to play with.
She's gonna get into them
And where was I? I definitely have my box of stuff.
I should throw away
False the ceiling. Oh, that's great oh.
So how do you keep track of what a page was most recently used
So you could use a timestamp for each page but that's too much data.
I mean we've got we don't want to be storing like 8 Byte per 4,096 Byte.
That's a lot of overhead we just want a couple of bits.
So yeah. So a flag
So you can use a flag. But how would you use a flag?
What does that mean? So the typical way you do this
So you just sweep, hey? pt's page table entry So so you you're sweeping through memory going through it.
And basically say, oh, everything, nothing's been accessed and then as memory gets accessed, bits will get turned to one.
And so then you know that those pages have not been acted so
There's one page table entry per page
The Page Table Entry is what stores the metadata associate with the page is not stored in the page itself.
Its story, and cause the page itself is bytes of the page. it's stored in the page table entry, so it's almost like the I know to data blocks for a file page table entry is metadata for
the actual page
It does this doesn't this is not freeing any applications this is just.
If there's some application that isn't running it wouldn't touch any of its pages.
But if it's running or doing it I think it would touch some as pages.
But that's how things get swapped out but it all depends on if this memory pressure stuff get kicked out, the first thing you do is pages that haven't been accessed that aren't dirty
So the dirty bit you can basically say is a little mark on a page, saying, We can't throw this away without losing data.
If you market as if you once you've written it to disk now it's not dirty if you threw it away.
You have another company that information on this somewhere if it was mapped to a file it just you know you've written to the file. If it's if it's not, you can still make it non dirty by writing it to
swamp. Now, of course, you don't necessarily want to write all of all the processes.
Memory to swap if there isn't memory pressure that's kind of like those dirty pages you probably don't want to bother file
No, the Tlb is a cache of page table entries
Oh, Martin, exactly that that's that's exactly it's exactly like a meter main that you've got it right exactly
Hey? stable entries are what makeup page tables.
Basically, page tables are just trees made up for page table entry.
The leaves. you could say other patient launching, but the internal notes are also could be considered.
Page table entries because of how they refer to the next level.
Okay, we're almost out of time here so you know wrap things up here, but these so this dirty pages.
Yeah. So we treat dirty pages that belong to file time that we want to process, cause you want to write the the dirty pages that belong to files directly disc as soon as possible.
But ones that belong to a process you probably want to keep those in round using this clock algorithm being constantly be making sure that there's a pool of pages, a large enough pages to be freed up so we are because if there
isn't. Then: yeah, problem, yeah, is it
No out. Ill
It is called the out of memory. Yeah.
Your Vm. went crazy when you're playing around the a four-code. So one thing, if you're loading modules in to memory, and you're doing memory allocation, you can get a situation where you start using up physical
memory,
Yeah, the automatic killer is drastic action. Yes, exactly.
Now is there potentially some way of avoiding? You know
Yeah, yeah, there is when the outer memory killer that, if you start reading up on the outer memory killer is a lot of controversy over it. there are potentially ways to adjust how it operates basically Once it kicks in you're just
in a world of hurt, but honestly, you're typically in a world to hurt before that.
But yeah, anyway, we're out of time so we will see you next week.
Good luck with assignment, for if you have questions, please go ahead and post it to teams, or you know, bug your tas and you know. talk.
Go to tutorial. You know there's still one more tutorial here next week, so go to the tutorial time and get help awesome.
Take care of