Okay.
Good morning, everyone.
I'm
so I was curious to see so y'all see the whole that I just started.
So not many of you thought was pretty easy, a good chunk of you thought the think assignment for is pretty straightforward. Some of you, it's hard, few of you didn't spend much of the tutorials I'll just say, if you didn't spend much time on the tutorials,
it makes sense that the assignments are they're hard.
Let's share screen here and. Okay, let's do that. The silver.
Here, we get structure.
Hey, I saw this I'm happy to talk about some things for the assignment give you all some hints. And then I want to talk about virtual memory. So let's see now so there's some questions about.
Okay. So, question five.
The key to understanding question five is to understand what it means to have a file with limited.
With limited extent.
So specifically, what you want to look at, let's talk about tutorial seven course this is where the module stuff is, in particular, look at new get PID specifically.
Notice how new get PID works. Now first of all it does a little bit of a cheat because you can't do. You can't read. If you basically seek it all into the file even a few bites.
It gives you is it doesn't give you anything.
But it does do this thing of, you know, make sure that the message is less than the size of the buffer puts the message into the buffer, and then updates the offset.
So you need to make sure you do all those things. So, logically what you should think of with this 10,000 by read is most buffers are going to be smaller than 10,000 bytes.
Traditionally, well, they could be larger but you shouldn't work for a smaller one so if someone's doing a read with, say, a one k buffer, what should happen is is they do a one k buffer, they read it, they get 1024 characters ones.
And they then they've updated their offset in the file to be now in the thousand 24th fight because they got 02 1023. And then next time you give another thousand 24 and you keep doing this until you've given them 10,000 bites at that point, you should
only give them, whatever, you know, whatever's left, you know until. Well, you know, if you do this, you know, with one k buffer, you can do this nine times.
And then the 10th and the buffer on the 10th time, you should only give a partial one so you have to do a computation, based on the current offset.
And,
you know, based on the current offset how and the size of the buffer, how many bytes to give them.
If you can do this, you understand the read system call.
Ah, is it acceptable if we got it to work without using the offset use some global variable instead No, no, because it's not acceptable that because then, what if two programs read from the file.
You know, you're reading the file no no no no you should not be using global year.
Are you supposed to change one seat you're supposed to create a new version of ones that works this way.
I still want a new get PID re Yes exactly, but you gotta change it around a little bit so that, you know, you have more, you don't, you can't do this.
This will not work
to global offsets. No, no, there should be no Global's.
So it's not so one key thing with kernel code is there things you can do to hack things to get them to work right to work in the moment,
but they'll only work for limited conditions and in particular.
You should be very careful about using Global's. Now, you could say this is a special device that has only 10,000 characters, and if any program reads from it, it gets exhausted so like you're treating it like a tape so that there is a way to that but
that's not what def one does so
that mean supposed to act like a file that has 10,000 characters.
So, as a pseudo recursive algorithm.
It's like what you've done here in in next get p idea essentially what you've done is you've made a recursive with a base case of.
Therefore returns zero, right. So, when this. So, essentially, if, if I can see what you're talking about it being recursive, and I can kind of see the analogy, but really what this is supposed to be like is reading a file.
And I, you know, you read the first meet. If you conceptualize reading a file is a recursive algorithm then sure I kind of more Think of it as you read the first bit of the file and then you read the next part of the file that you read the next chunk
the file. But in this case, I mean, essentially, it will read either until the buffer is fuller until zero is returned right because that's you're setting the offset and basically you're counting down on the offset is what you're doing. Correct. Isn't that, as I've
Isn't that, as I've understood, you're counting down the offset correct. Now, is this the only reason I can get away with this this is this you know the offset greater than zero here returns zero, that's wrong for 10,000 ones because, basically, I can't.
Why shouldn't be able to do if I treat this as a real file is if I had an offset of for.
I would just not output your and I would give a space p ID is whatever.
Okay. But that's not what this does because I was lazy here, but for the 10,000 ones you can't be that lazy, you have to do the calculation and say, What should the offset the.
So my current offset, am I, more than, am I, more than 10,000 characters, you don't return anything, because there's nothing past 10,000 characters. If you're.
If you have an offset that's less than 10,000. You then figure out well how many bytes or left is that more or less than what's in the buffer you figure out what the length is calculate that, and that's how many ones you return.
So it's just a couple of calculations. Me, it might make sense to draw it out in a piece of paper and just kind of think about the different cases but this is what's happening every time you read a file so that's why it's worth, you know, thinking about
it.
Um, it's literally just a couple of statements.
And the other function gets called okay.
So there's a question. Okay, explain how the read function gets called So, question, how is this getting called.
So this is all about. Whoops.
Turn my video there. Um, So what this is about this goes back to hear what you know when we make a call to would have ones the ones read function. So, when you implement a special.
A special file. It's like a mini version of file system rather new for lots of different files you only do it for one file but you're giving the file operations functions for that file.
So you're saying when I do an open when I do a read when I do a right when I do these all are you implement those as functions and so when you go through the system called stock you can actually do a trace on you get big read and you can see all the file
operations leading up to it, that are being called.
And so that's what you, what you want to see. Logically, what's really happening when you make it you know when you make a system called
what's happening. First off, hold on, we switch windows here.
What we make a system called what's happening first.
So this is the logical flow of control that you should be thinking of when you make a system call is the system called makes a Cisco instruction which is causes the CPU to go into supervisor mode and causes the entry point of the curl specifically, the
system called entry point to be run. That's, this is actually.
So there's this assembly Glue Code, that gets called. Once the system calls done you know when this is called requested a guy goes like what's this called things that's manipulate the registers.
And then as quickly as possible it switches back to see and calls the appropriate function that we want so all those Cisco defines.
Then when the system call code returns, you know, a C code is just doing its thing. Then when it returns, then you go back into use basic mode, via the scheduler and the schedule is partly and see but of course the last little blue parts of the assembly,
and that part just kind of decides, oh what process should my next run and does a lot of calculation decide okay what's next priority which one Chiron and make some decisions so,
so, So notice that when you make that system call.
So if you ever want to know who made the system call, its current.
And that's, you know, type stuff.
So do we return to you space after the Cisco code has been run Yes, you do via the scheduler. So what that means is the process that made the system call is not necessarily the one that has returned to user space, you may run another.
Yeah, I just wrote what type it is.
I'm kind of creepy i think you know bite New York Jets.
I'm
so confused so Shay You said you're confused about CPU registers.
So,
so CPU registers, I'm.
So this is called a load store architecture.
You load values into registers for memory you do operations on the registers and it's a direct memory.
And so when you make us know.
So,
so when we mix the recall the three. So, the key thing to understand about looking at this function is remember.
Okay, so.
So, do all hardware register share this functionality.
All hardware, direct share this function out so this is what. So, think of it conceptually like this memory RAM is circuitry that allows for two basic operations, you store data, and you retrieve data.
And because it only allows for those two operations, the circuitry can be very uniform, and thus can scale up to large amounts of storage. That's how we have gigabytes of RAM.
But in the CPU is not enough simply to store data. We have to be able to manipulate it in order to manipulate that requires additional circuitry.
And that additional circuitry has to work on specific bits, those specific bits, or the registers.
So, you think of registers as just RAM, but with lots of other stuff around it so you can manipulate that state.
And that's why there's only a limited number of extra them you go back to older processors like the old six five or do only have one general purpose register the accumulator.
That was the only one to do general arithmetic and anything. And that was only eight bytes and eight bits. It was really small. And then you go to current systems we have 6464 bit machine means that you have 64 bit registers that you work on things that
64 bits at a time.
And you have like, you know, at least, 2030 registers and actually under the hood, more than that means you have lots of registers that are general purpose that you can do all kinds of computation with them so you can store, lots of variables in in different
ones. Some of some registers are kind of like, you should always keep certain values in certain places, there are certain things but oftentimes those are conventions the CPU can just do stuff and a lot of the art of making compilers is like, oh, the user
described, you know the programmer, define this variable, what register should I put it in.
Or do I have to kick it off and put another one in.
So, um,
ah, so one of the key things that has to happen when you switch between processes or switch to kernel space is the registers have to be saved and unlocked and loaded.
I've not really talked about this.
It's a very key thing.
If you, if you start learning assembly you'll see this. But yes, Robert you're exactly right.
So let's go back to let's let's keep going here.
Okay, so
they are top file operations to this.
So now, so questions. So,
I have a little error with this, we fix this.
What I'm asking here about is understanding what the heck is happening in in tutorial eight with the shared memory.
So, what you should understand here is when you look at this data structures enough.
So we have this data structure here.
And what we need to do is allow two processes to access the shared memory. At the same time now, in a multi threaded program, which all memories be shared but you would logically use different parts and you just say those two threads okay here's some,
know, here's this data structure we deem is being shared here, I'm saying most memory is not shared, except for what was explicitly decided was shared.
And it's this.
So, with this we have, you know, what do we have here, we have rq.
And we have some stats over who's the producer who's a consumer whether process IDs. What was the Latino, which, you know, which, you know, entry in the queue was last consumed the last produced.
How many of you produce how many consumed. But then, we have these, these are the variables that are enforcing mutual exclusion, which means. Well, let's be clear what sort of mutual exclusion is being enforced here specifically what what we have to do
here is,
this is, you know, this, this new text basically says, Oh,
this is saying, We are manipulating some of these conditions.
So this right here actually does a lock is a lock on both of these here, which is saying is this one in or are we messing with these values. And the idea is that you grab this
away so this is the wrong one. Let me do this one.
This one's better.
Yeah, thank you.
Yeah, cuz it yeah thank you this this is the one.
Exactly. You have to lock before single band condition variables. Exactly. And the reason why you do that and that's actually part of the reason why the other one is wrong, is we have to condition variables and basically what the condition variables are
is they serve as a kind of messaging thing, but instead of a, we couldn't have a signals, which, you know, I've had, I think it's not an old version of this that you signals, but you could have a single between two prophecies that send messages you know
say send a sick user one saying, Hey, wake up and have them send that back and forth each other But the trouble is that goes to the colonel here this purely uses memory, it doesn't involve any system calls, and these two are now basically going, Okay,
this basically is a message like the.
If this is the event non aq non empty.
So, if the,
if the consumer, basically goes like, hey, the queues empty, there's nothing for me to do, then says, I'm going to wait for the queue to be not empty.
And so this is what waits on this condition and then the producer will then wake it up and say this condition is met go wake up Go do your thing, and reverse for the non fall.
But we have to worry about these things might.
We do want to make sure that we have exclusive access we don't want to two threads or proxies accessing these variables at the same time. So we have to protect these using the new taxes.
And these new taxes basically are saying, you know, I'm basically putting a lock I'm saying, I have exclusive access to this variable during this.
So know that dash timeout does not deadlock. It's, it's fixed it's the non timeout one that doesn't deadline. And in fact, if you look at the conditions that you need to deadlock.
What it means a lot of people make a lot of big deal out of things that we did deadlock. And there's all these conditions you can look in the textbook which says very nicely about what deadlock is, um, but the key idea of deadlock is, you have two or
But the key idea of deadlock is, you have two or more things waiting for a conditions that can never be satisfied because there are there's a dependencies between them and one way and the causing this happens with walks is like, I need like a and b, you
know, you know, I need lock x and y and you need lock x and y. And we're both. I have lock x you have lock why I'm holding on to both of them. So, neither of us is ever going to progress.
And so we're, you know we're kind of in trouble here because we have both of these things.
The question there is, how do you prevent that from happening. And a key way you prevent that by happening is adding timeouts because basically, I have, I have walk x, you have walk why we're sitting there waiting, nothing's happening like well I should
give up my walk.
So, it's that, if you have infinite way, if you hold on to locks and you never let go of them you deadlock that that's one of the key ways that locks so well solution is, you can hold on to a lock you hold up basically hold on to resources.
Don't ever hold on to them forever. If you're not making progress, give them up, let someone else use them. So hopefully you can try grab it and then try grabbing it later.
So
me really what this is what we're protecting here is we're preventing to basically the two of them from modifying these variables at exactly the same time.
We don't want that to happen. We don't want these to be modified exactly the same time because then there is that so that's part of the P thread API is you have to give them you text to do mutual exclusion on these.
But mom and Are you, Are you still confused, Andrew do.
So the idea of so the way you're avoiding a race condition is just that only one of them is accessing at the same time.
why we need a new text wait and P thread asleep. What are you talking about, I'm good Gordon whittling.
So, if you can. So one of the things is, this is where
this is where I actually pointed out some stuff, if I actually know what am I doing here I actually have this in the tutorial. If you looked at the top of the tutorial.
I say condition variables Semih fours.
Read these.
I'm happy to answer questions about them but this kind of goes into full, full details about condition variables, and some of fours and things that we don't really have time to go through them in complete detail there's this is a.
There's a lot to talk about with these. What I will say is that, in modern concurrency programming you shouldn't be using these because these are way too primitive for much the same reason that we don't spend a lot of time learning assembly language it's
way too low level, it leads to way too many problems we have a higher level constructs for doing these, but they are language specific and they aren't there and see so we're not really covering them here.
But if you learn languages like rust or basically any modern language they have some thing better than some force and condition variables for for doing mutual exclusion.
So the idea is when the timeout ends, it gives up the lock.
And then, let's make a cue basically is kept for who else was waiting for the lock. Because what's happening here is when you do this.
When you do a.
Let's see here. Here you go. So,
we do this wait for producer code. What we're doing here is this is the wrong version.
Yeah, so when you do this wait for producer what we're doing here is we set a timeout.
And then so we grab the lock. We tell we do the P thread your time Wait, this, we give it the lock that we have in the condition variable in the timeout.
And what it does is, anyone who wants this condition very basically anyone who wants to access to non empty, Will.
Will that will actually sorry, this is, this is just waiting for the event to happen. And so you could have multiple ones waiting for this event to happen.
But in, but behind that the mute taxes, it will keep track of who's waiting on it, like if you're if I want to access this, I'll go like, I want it. It goes like you can have it.
Wait, wait, wait oh now you can have it.
The timeouts are really happening. The timeouts aren't affecting the condition variable, they're affecting the new tax.
So
what's different CP thread Conway and some weight.
Okay, so, several way is for waiting on a semaphore Conway is weighing a condition variable. The key difference between some force and condition variables is conditioned variables is for one thread or process to send a message to another one there for
sending messages, some way is to say, I want this resource is it available. Tell me tell me okay it's available.
So that's your summer forest for exclusive access.
So that's the way there is waiting involved with both but but it's for a different thing because you don't mean there's an implicit kind of messaging happening with the summer for but it's just the resources available or not, it's not that.
Hey, this happened.
Okay, so I have not talked in full detail about mutual exclusion and things which is one reason why I forward it, you know, talk to you about these and my questions about these mean that's why you'll notice in the assignment I'm not asking a huge amount
of this but if you can understand this.
You get the basic idea.
And the basic idea is that you have, you use some of fours, to pretend it access to.
With this load store architecture.
It's really tricky to have one variable be accessed by two cores, because that variables and main memory can be in caches and can be in a CPU register, it can be anywhere along those lines, there's potentially multiple copies of the same variable all
over the place. Logically, you're not thinking about the CPS to pay attention to these things. And so if you have to, to course right to access the same thing, they have to do some coordination, to make sure it appears to be the same thing and that's
why we have to use the special operations.
So, the condition here.
So the resource here is the Q. No, this is not the resource here is not the key, the resource here is, is actually just the.
It's actually just the condition variable that So, look, if we look at this.
If we look at the where we are tutorial up.
She might think about why don't we have.
a look at this, we only have two new taxes. And these two new taxes are protecting these two variables.
Why don't we need to protect the entries in the queue.
And it's rather simple.
It's because all the.
It's because of how things work. the producer will only take an will only right to an entry that is empty.
And the consumer will only right to an entry, that is not empty.
So they check it and if it's in one state, it will convert it to the other state but the only both go in both ways. So, so they don't actually cooperate each other on individual entries in the queue, where they do get messed up.
And the reason you need these is it's very easy for one to run faster than the other, and kind of go through the whole Q and just go like, oh I produced everything now what do I do, and that's actually what we're implementing here is the sleep wake up
up here. We're not implementing mutual exclusion on the queue entry.
Because it's not needed.
Yeah.
Yeah. Yeah, that's it. Yeah, so, so the idea is that when we. So you go through the producer goes through the whole thing. And then basically goes, Oh, it's up to you fail fail fail fail fail fail.
And then once it's filled up the whole QE can actually often Philip Oq before the main be before the consumers even started.
And then it goes, it comes back around it goes like Wait, it's full.
I have nothing to do.
Tell me when there's room to produce.
And that's it, and that's it you know basically it starts waiting goes to sleep and waits on the message, basically saying that the queue is non full, and then the consumer goes and looks at the queue goes like hey, it's got stuff, let me go boom, you
know, and starts doing that.
So, um, so you so you can go, you can access this queue without accessing these variables, there's no exclusive access to these. And in fact, you see this in the code.
The animals doing out there are those the cats like the dog is fine. Um, so you can understand what's actually happening is, you know, this is the heart of things.
So I get next word.
You know, this is saying this is what the consumer is doing.
So first, oh sorry we do have a lot.
I'm sorry, I'm being stupid.
I do have more locks, I'm just I'm telling you garbage you
Because if you look at the Q.
But every entry as a summer for.
I'm sorry I'm being stupid
sure what I just said, Get the past fight.
Oh goodness.
Trust the code.
So let's look what is happening here. We first check to see, you know, this is where this is where we are currently in the key. That's the current value.
And then the entry there.
You know, so current gives us basically the index of the there, we get the actual entry value which is type entry here, we lock it. To make sure no one messes with it before behind her back, we mess without do all things we go like, okay, You know, if
we do this, there's no word. Then we got to go to sleep, and wait for the producer.
And if we do it again and there's still zero after we have the producer. Our codes messed up.
But if it's now there's something there we copy the word do all these things and do this. No go through all this stuff. And then when we go to done, we make sure we release the lock and tell the producer, that we did some work, so you should wake up.
Is the summer for over there so we can have multiple producers consumers. Yes, exactly. That's what I was forgetting is that I had put in that implementation.
If you only have a single producer and single consumer this level of locking is really not necessary, but if you have multiple producers or multiple consumers, then you really do.
Remove the locking for the condition variables I get no nothing for consumer Yeah.
Yeah, that makes sense is a non full signal of the item only the
is a non full signal on that item or only for it's basically a message. When you say it's non full.
What you're saying is there's a least one empty entry.
You're saying there's no, there's, how many there are you saying there's at least one non empty empty and well how do we know this because we just used it.
So, question is how, why do we use a go to here. This is actually a very good question. It turns out you look at the Linux kernel. There's lots of goatees.
Go tues, though and see are only within the scope of a single function, they can't jump between functions, and because of that they actually can be very useful.
And it's specifically what you use them and see typically is for error handling.
So the idea is that all these things here all basically saying, I'm, you know, I definitely want to go here.
Where do we
honestly with the way this is structured, you don't need these go Tues.
I think a previous version of this have go tues here, and I had other stuff going on here. This version, it's just going to fall through the conditioning though here, but the reason why you often structure things like this if you look at the kernel code
that's part of the reason why I did this is because it's not the kernel code does is you grab a bunch of whenever you grab a bunch of resources, basically grab locks, you have to be very careful to release them before exiting the function.
Because if you don't, you're in big trouble so there's typically a bunch of clean up that has to happen and so you may have multiple paths to your function that could exit.
And so by having this, you make sure you, you have a block of code that cleans up on exit to the function and does you ever go to to say, Go clean up there.
Um, if you look at, um,
let's just go to Fs.
So let's go to read, write and you shouldn't be.
This is one that we.
So like here, this is Elsie.
Um, You'll notice that here is out.
And what is it doing it's releasing locks.
And then returning. And so you'll notice, instead of returning we have this oh there's an error. Go to out because we got the locking of but basically here, we have a lock here so we better release it so go out, go out, go out, you know you have all these
go tues to make sure you release the lock.
And you'll just see this all over the Linux kernel. Oh look, here's an earlier release, you know, free up the data structure, before you click No, because you're not gonna need it so that's.
So it's all over there.
Yeah, yeah, we're doing non atomic operations.
So, the reason I was a little flippant with this is actually these entries in these choose of these arrays, these arrays are pretty small, the fact of two processes are the two of the two process threads, accessing the same word at the same time without
having finished it is actually pretty low. But this is the thing about concurrency.
If you do something, often enough.
The low probability events happen sometimes, and that's where you get these stochastic bugs were these bugs where most of the time, it works fine it works fine then all of a sudden, boom everything blows up.
And it can be held to debug.
So yeah, you put the lock around each of these things. And then we have the condition variables to basically, we had a new taxes to protect the condition variable so they essentially every piece of data here is protected by a new tax each condition variables
protected by a new tax every word is protected by by me checks.
But here.
This is actually.
I forget why you send, I think I use some teeth I could do down and up. Because you can have multiple people wake up to remember exactly.
The idea of a summer for, is it can be accounting summer for you can have multiple values with a new technical have 01.
Again, way more than I can discuss here. I'm just giving you a flavor for it. and you should understand at a high level.
Don't need understanding a detailed level.
The why each entry is is because one of them
is basically because one of them could be accessing the word So why do we need them you text on each of the entrance is because it's unlikely to happen.
But notice that when we do the string copy.
This is the keyword. So this we're doing here a right.
What if, while we're doing right.
Someone else starts reading it exactly at the same time.
Before we registered that this has happened.
You're going to have the situation where this one rights to it exactly the same time the consumer reads from it, and you get this thing where they never actually update the accounts properly.
And so, things break.
I will tell you gain this code working right was a pain in the ass.
Simply.
The condition variables are just there to say that is this there to allow sending them messages, they don't provide mutual exclusion.
Okay. Other questions on the assignment.
Otherwise I want to talk about
was talking about tutorial nine.
Because that's so tutorial nine isn't directly in there in the assignment, except to.
It's kind of in there but not directly, I'm quite sick see you want me to ask about 60 okay so
the next lecture will be a review session for the exam because I'm going to go over the site. Basically, remember, The exam is based on the assignments.
So they going over assignment for and then go you know they spend the rest of the time reviewing for the exam.
So there's a question about six on the assignment,
can we create a file that will make the modified 3000 pc FIFO behave like the original
either not to give a hint on that, without actually just telling you the answer.
So I can say,
you. Yes, I saw I just said, Yeah, so I said that the, ya know that so i'll be reviewing for the exam on on Thursday, because I don't think going through the solutions will this will take most of the class I think it will probably take about 15 minutes, and
then we'll go up and the rest of time reviewing.
Okay, I'm
okay. So one question here is this.
I will tell you this.
All you have to do for this one first. This again I can't explain too much. I will tell you that if you make this change to the code, there's the question of this to make a change in the code.
If you make a change with this in the code. The module will behave differently. I'm not going to tell you I would behave differently, but it definitely behaves differently so specifically what I'm talking about is replacing the put user code with this.
If you don't see a difference you aren't using the character device.
Be sure to look at the colonel logs.
When you do this, just like for any module it's out putting things, the colonel logs will say stuff.
Okay, so.
so I can't, I'm not going to say more talk about mutual exclusion because there's me I can spend the next couple of lectures talking in detail about some fours condition variables I really give them short shrift here, but it's just, They're important
in a low level sense, you should know that they exist and about them but good lord, try not to use them. If you find yourself using them study them.
Um, what I do want to talk about now, though, is this tutorial I mean, I'll explain the whole killed and things again, it's all the pieces of there from a tutorial but
no, ask me questions about the tutorial I will answer those I'm not going to answer anything more about the assignments.
But tutorial nine.
So what is tutorial nine doing and why should you play with this. So what this is actually showing you is, let's just let's just demonstrate this for you.
Okay, so now let's running this is actually pretty simple.
Notice I made this a terrible.
I don't have it downloaded.
OK, so now to use this.
Now you run this, so basically what you've done is,
We made a device here but this device is a little weird. It does input and output. So you if you want to figure out how it works go here but the thing I'm actually interested in your learning from this tutorial is not necessarily how this works, although
it is worth your time spending a little bit of time looking at how it works but it's a little complicated, but what I really care about you understanding.
Is this
it looks more structured for physical memory is probably because obviously know why that is but it made. Yeah.
Just let me I probably haven't done much on this young, but what you should understand about this is that the mapping between virtual and physical.
The, the ordering of things in physical memory has no relationship to the order of things in virtual memory, specifically, what you will notice is there is certain commonalities and these addresses the lower several, you know things are always the same.
You'll notice that the, so this is for six c zero. So six co so the last three is always the same which makes sense because three each digit represents four bits.
because the value from zero to 15, four bits.
So three of them is 12.
So 12 bits that allows you 4096 values, ie 4k. So, so notice that the last three digits of these are always the same between the virtual and physical address, but the, the rest is completely different there, and they, they're mapping collective ways.
Now, question is, why would you see, after I run this multiple times.
If you look at, you know, main here is this.
And then I look over here. It's exactly the same value, why is that it's because the binary have three of 3000 3000 members to has been cashed in RAM between executions.
It's been saved in RAM, at a certain visit in a certain bit of RAM, so it stays in those pages. So you'd have to flush RAM, you know, flush him getting to kick everything out for that value to change.
It just gets wherever it's a free space.
Tom using Oh Lord,
so.
So yeah, in fact if you did, 3000 if you basically look at the virtual to physical mappings for.
Yeah, so if you're running, you know, multiple versions of bash, and you then look, they will be the binaries might be loaded as a different parts of physical memory, especially will actually they will be because of address space layout randomization,
that's the whole thing it does is it moving around things in virtual memory.
a whole thing it does is it moving around things in virtual memory. But the physical memory will stay the same. And in fact, it will stay across different programs, because the different programs are using the same physical memory.
So, if you have repeat physical mappings. It means it's stored in RAM, in the same place. Logically, is that it that it could in principle be pinned to memory so it's not cash it's like in there.
But when we say cash it means that it can be, you know, capital removed as needed.
But so main and the put us from our cash, but the things like it called when we run the process like arguments or dynamic allocations are in visit, physical locations.
Exactly. That's exactly what's happening. So you'll notice the, all the stuff that's dynamically allocate it all the data portions.
Those are going to be new each time you know and you might say well why didn't it just reuse the same page as well.
Think of it this way you might have a page that is, you know, a bit of memory, that's all zeros versus another one that's got some garbage in it, which you're going to use well if I got zero pages I might we'll just use the zero patrons.
And let someone else's you're out those, the other pages.
A.
So, notice the mapping happens in everything but the bottom 12 bits were literally mapping.
So this is, you know, you would say that, you know, one terminology this this is the virtual page number. And this is mapping to the physical frame.
But, you know, the last year, everything here so everything except the last three digits. That's what's getting mapped by the virtual memory hardware is it's not literally every time you do an address look up, it takes this and looks to see what that's
associated with, and then says, okay, its associated with this, and then copies over the bottom 12 minutes.
So, so I'm not sure, Mickey what you're asking this virtual memory different for L message in G message. I do you mean virtual memory of what this is all about virtual memory.
Well the last three digits of the offset of the page remember.
you just copy the offset doesn't change because it's just an index into a 4k array.
Every, so remember everything is in RAM.
So when we run top here, and we say what's.
You look at what's happening.
You'll notice that we have this buff cash, just kind of says this thing, this is really just all the pages and what's stored in there is here is you know all the, all the stuff in the process and all the files that you access.
When you access a file, you're reading it really directly from disk, what's happening is a file half portion of the file has to be loaded into RAM, and then your processes given access to it.
So, that, you know, that whole back and forth, you is done by the operating system transparently, but that's what's happening under the hood with all the virtual memory subsystem and effectively.
The more RAM you have, the more space, the operating system has to work with for caching things from disk.
And that's why you can often have better performance typically RAM is faster than this and so these are good at fast and all spinning hard drives. But still, they're still slower than Ram.
So,
I'm.
So, the idea is this, so, so, so,
so even the kernel itself uses virtual memory because using physical memories that pain in the ass. You don't want to be. You want continuous ranges of addresses, and you don't want to have to lock down ram that way so most of the colonel is using virtual
memory.
But its own mapping is effectively the colonel acts like its own its own address space.
So really,
Just think of slab allocation is a fancy mouth.
cash it Yeah exactly, yeah.
So you want when you use these data structures, you don't want them to be like if I need a task, where I file structure or any of these structures. I don't want to sit there and out to initialize it and if you actually benchmark.
Regular programs.
It turns out that dynamic memory allocation is pensive.
dynamic memory allocation is hugely expensive, it's a bottleneck and any complex program, so complex programs almost always use.
It's really this is saying if you looked inside Firefox you looked inside any sort of complex program.
There's a lot of effort made to make memory management, efficient, and there's lots of strategies for doing this.
It's a complex topic.
But the Linux kernel to be efficient has its own memory allocation things basically to make it so that, you know, you don't have to spend much time allocating things are so bad.
Oh, what you want here it is, though, there's there's your thing here's a pointer it doesn't have to spend much time figuring out where things are.
If you're making a simple program just using monologue is fine.
But Malik is slow, which is the reason why. See programs try to use the stock wherever possible because the stock is hella faster than doing things mouth.
Yeah, it's not about once you have the data structure, its memory it's going to be me. Performance doesn't depend on other factors, which also can be optimized in various ways.
It's all about allocations.
You know, it's getting things I want to allocate this memory here.
So,
it's garbage collection in Java to prevent using Malloch. I'm garbage collection is a strategy for memory management.
Now block is definitely is definitely
not know like is really stupid as a memory alligator. It's a really dumb one.
Oh, no complex program just uses Malak, they might lay or something on top of Malik or do other things but Malik itself is
garbage collection is a custom memory allocation strategy used for dynamic languages.
Java uses garbage so.
So if you don't have free, you're probably using. So, so if you don't have free and you have data structures that last beyond the invocation of a single function, you're doing garbage collection
pages like a block but for exactly yeah it's really. Yeah, that's exactly, That's exactly.
So when you do page allocation
garbage collection is strictly slower, but it's.
garbage collections a tool thing but it's it's way outside of sports.
Yeah, it depends on what was optimized enough there's. But, um, you use Malik because it's simple. Exactly.
Because you don't have very complex needs and typically we're often Malik is used to say, give me a big chunk of memory. And then inside that big chunk of memory, I will make my own allocate or do things.
You know, I will split I will chop, like, give me, you know, a megabyte of RAM, and then I'll pop divided up into little 10 by data structures.
Metal memory is efficiency, how you use memory isn't a huge effect on efficiency.
So, so why so.
why do we use pages because that just makes everything easier we just allocate things in 4k chunks.
That's it.
Um.
Now of course the real world you know things don't fit into 4k chunks but that's why we have the abstraction of virtual memory, the effect the raw memory we just manages 4k chunks we don't care we just assemble some the hardware know that's all there
It has its own dedicated allocation scheme which is built on top of pages underneath.
We care what level is but it's just like, Oh, I have this area for my file structure, I have this area for my cost structure by this year for my cash and just divides it up into areas on memory management in the Linux kernel is complex and changes all
the time so anything I told you I've looked at it in the past I know old versions of it, it almost certainly isn't valid for the current ones call.
This is, we're talking about
Linux is, I mean, any modern piece of software is very complicated more that can be possibly covered in one semester class.
The purpose of operating systems is not so much to teach you how everything works, but to get you aware of the right questions, so that you can learn what you need as you go.
I'm even 4000 would be before the account 4000 is actually a completely separate course by distribute operating system which again is to ask you the right questions, but not on a single host.
But on, you know, instead of distributed hosts.
So a block blocks and pages don't have to be the same size blocks on disk pages are in round.
That's all. Oftentimes the block size of the file system is exactly the same as the page size in RAM, and that's very convenient when it is the case. But that is not a requirement and that that varies but blocks on disk pages aren't Graham, and oftentimes
we cash blocks in memory, using pages
slobs are typically sub page things but they couldn't, it couldn't you could have the slab alligator could be things that are much larger than a page.
You missed the poll.
After useful question please.
I'm trying to anything else with you. So what's cool about this code is.
You can I show you that the virtual the physical mapping is a real thing. it's just governed by code. And in fact, basically there are these patients, if you want to look up page tables, The textbook has a great treatment of page tables.
This tells you all about the trees, you know, about how the page table entries in the multi level page tables and stuff and, but it's just code.
And we can figure out, and this code here is figuring out but again, I'm not going to.
If you want to go through this and try to figure out what's going on that's great. But the thing that you should really care about is this user space part.
A three I'm hoping to have all graded by Thursday.
We're trying to get it all done by by tomorrow and so they have on Thursday. If it's not on Thursday, it should be soon after assignment three.
Um,
so why do dynamic collection slow down systems compared to collections, it's because in order to figure out where to store something in memory.
You have to figure out where to put it.
And figure out where to put it involves reading a lot of things in RAM in different places. And it turns out, accessing random parts of RAM is slow. Just walking a linked list is kind of slow because it's all spread out and modern memory access is all
optimized for x and contiguous things of memory so a memory allocate that time to look well is this being used as this being used as this free is this available.
It just takes a lot of work I mean and you can amortize it in various ways and say we're going to do it in the background or this but still it's gonna take time to just manage it.
And, frankly, it's, there's no general so the best general solution, people have come up with from memory management is garbage collection, and even garbage collection is performance, or it can vary hugely dependent on the workload so then fact there's
lots of work on different kinds of garbage collection.
It's a complex field, it's fascinating because it says a lot about how programs behave, but it's.
Yeah.
Yes, it's faster to look at the next go to go to the next indexing. Yeah.
Yeah. Same for us midterm questions I haven't finished making the midterms I can't say exactly. Um, but it's probably will be proportional.
You shot for five minutes per question.
So it's two hours. So hundred and 20 minutes.
So, it has that workout.
Hundred and 20 minutes.
So I had 10 questions so like 20, something right 4824.
Yeah, there's a lot of questions but again it's covering the entire semester. So, I have to ask all, it's still online open, and I will be. So I will be trying to do interviews.
I'm hoping to get things graded.
But I, so interviews will probably be during the last week of December.
Just
so probably it's going to be after Christmas because your exams could be like on the 16th.
Um, it'll probably take at least a week to grade, so probably the week of the 27th will be for interviews and if necessary, I can do things even after I've submitted grades just to verify things in the seventh or something but now I can be great, I can
do great changes if it turns out on your great goes up.
Well, you now know what to expect.
And I will just say that make sure you study the mentor.