Let's say yd is that the end of this class. Okay, let's get started let's put the recording on.
Okay.
Good afternoon.
Good morning everyone.
So today we're going to be continuing to talk about tutorial five.
Orioles.
So, so I want to show so I showed some things with sh Fs.
Last time, but I didn't really go through the mounting file system stuff the way this tutorial does.
I'm going to really I'm planning on releasing tutorial six to, you know, later today. So you all have something to play with there I need to think a little about what I want the next tutorial to be.
But I want to make sure you have played to work on.
You know, when you have time.
So I've got right now I'm on the class VM, so I can make sure to do things, although let me, so one thing, it's always good to do when you're running these VM.
So I'm just doing this.
Doing an update here basically so app get update to update your VM, when we go on this.
Yeah.
So, I will talk about snaps
backup before update.
It's good to do backups.
Move your terminal a little bit, please.
Is it is that ok for the terminal.
Like I can reduce my screen sharing size a little bit when we make this.
This to be over here, let me just.
Can I shrink you know I kept shrinking that way I can do this.
How's that for everyone right now in terms of screen sharing that look good.
So the reason you update. So
what's.
In general, when you looking for packages.
It says, user share.
See here, this has a
lot less.
This I operate Packers automatic on attended if you're part of disable it from the command line run this. So, like any package you want to do.
Couple useful things.
The reason why.
So, so there is.
Georgia I sent out install. So we got app install out move out purge. Yeah, these are kind of your basic app thing I realized I never really talked about that because we have any good but you may actually have some reasons to do that.
I'm also there's.
Also there's.
So, you can also just search, you know, for if you're looking for a package. So
ups.
right, then.
Then,
so the reboot is only needed if you update the colonel.
We're gonna be talking much more about kernels, very soon. Anyway, where was I let's get back here.
So I'm now running on the VM here, logging on to the class VM.
And I'm on, it's up from here so now let's now make or make a directory.
So my tutorial five directory. And so, let's, I want to go through some of these things here and talk about what's actually happening with these things so we've already done, you're seeing LS dash LA, I noticed that we have I nodes associated with directories
as well. And so, you know this for nine, you know, for one zero for one on 190. That's four dot. And so,
Here, you'll notice that if we look at what's the number associated with tutorial five is 410190. It's exactly the same as this 1410190. And then, this here is referring to parent which is home directory here, which is the doc here, which is 4101611.
Notice the number here.
This is saying this has two links to it.
What's, what are the two links, you know that you are links to it one is this dot, and the other is the entry here.
So all directories, have at least two links. This one, the dot dot has 21 links, which is saying my home directory has 21 links, why does it. It's because well of course there's a normal dot entry and those the entry for where it isn't slash home, but
also a dot dot entry for every directory in it. So let's see how can we find all the directories so one. Okay so 1-234-567-8910 1112 1314 1516 1718 1919 plus two equals 21, which is this.
So, but, So what does.
So just for us going back links Oh, go a question, what is app clean do so.
Rock cash.
So if you look mr. cash.
Well.
was the transcript.
Okay, that's the problem.
That's annoying.
I think you can move the transcript but that that's annoying as that better now.
That's definitely not what I want to do.
There you go.
Good.
So I
ok so now we're.
What are we looking at now we're so let's, so we've shown you the I nodes. Just pay attention to those I know numbers that shows the identifiers. So now let's go through this run DD may and we're making a file.
So I'm going to to five.
And we want to. So now what is this command doing DD, so this is the thing you should be looking to understand is just everything that's happening. Everyone these commands us are thinking, What am I actually doing when I run these commands.
So DD in file equals zero.
So now I'm reading it in from the device dev zero dev zero produces an infinite number of no bytes.
You know, equals.
So now we're running it out to a file fu bs is block size. That's how we have eight k block size that's 8192.
And then we'll do count was 32 K, the amount of space. This will take up is 30, you know, 32 k times 8192.
And then.
So now, what is this here now.
I'm
an H for human readable so this is 256 megabytes.
Foolish is.
Okay so, if, if, if, yeah, so if is input for I was input file and of is output file. That's exactly right. So it's inputting from here and I'll put into here, and this is what it's doing so well a DD does is it copies from input to output and one thing
to note, if you look at the S trace of DD.
It is actually doing exactly the number of right system calls that you're telling it to do, which is very cool, because otherwise you don't have control most of you have control over this so what this is saying is, it will do 32 k rights, and each right
will be of 8192 bytes.
That's what they said. And what what will it be will read boil read 8292 bites from Deb zero and then write that to food, and it keeps doing that, that many times so if you do an S trace of this, you should actually see exactly that number you can try
the count of one or two, and see if this is right, but it should be for that app for that.
And that's where something to check, so d is DD similar to the,
the, you know, greater than read director, I'm
kind of all that when you say greater than, all you're doing is you're saying that before you run this next program
with before you run this next program, you've got to the shell should open this file and assign standard out to be that file with DD you're telling DD take this input file, take this output file.
It's not doing it for another problem DD does itself.
And it's a read from this outfit. Now what does it do if you don't specify these things, dd reads from standard input and writes the standard output. So you could use DD, and then pipe and things there but you want to do it this way, normally when you
do DG.
Yeah.
And, of is out. Yes, exactly. Um, so we got this file here. Now, let's look up.
So, this file right now is taking us 256 megabytes is taking up 257 megabytes with a little larger
up.
When would you want to pipe things using DD.
The reason you'd want to pipe things using DD is if you're using a device that really wanted things written in blocks, like you didn't want to have.
You didn't want let's say have something that's more diversity in this case and lots of data than have small little bits or something when you need it chunk up.
That's when you use DD. It's classically used for writing two tapes.
Nowadays aren't typically a lot of reasons to use it for that sort of purpose.
But we have this file here is taking up the space. So this is so if we just look at the contents of.
It's just no lights.
That's what
control see
guy killed it dead.
Stephen remember with dd dd has a program that is called that's whole job is to copy from its standard input to it standard output.
When you use the greater than operator, you're just saying, open a file for standard output. It can be used for any program, but DD specifically is copying from input to output.
So the closest equivalent would be if you wanted to use cat and give it, you know, and, you know, redirect some file from standard input and then have another file stereo output.
Did they kind of be like that but you wouldn't control the blocking.
Oh,
OK, so this is actually an interesting question so so Adams asked them if we, if we say DD if we'd have this file of the program here,
are we setting these as defaults.
And this is a.
So, if we ever enter anything into a command.
Is this in general a default for a future command.
No commands and Unix are independent.
The only time they're not independent is they change something. And the thing they would have to change as either a file on disk or an environment variable.
So know that you have not set a default.
Let's keep going through here. So, run. Make it fast ext for on food.
So this says make a file system of type the XD for into the file, boo. Now remember, normally,
you know, normally this would be a device, but this will also work on a file.
So, what is dev zero. Check out dev zero. I will tell you it's a device that gives all knows.
Let's see if I can do this.
That's what dev zero outputs
worship.
Huckle, you know, so if you SYZNX is out putting zero bytes. That's all it is.
What's the purpose of dev zero exactly this.
To give you a zero bytes.
That's it's only purpose. Turns out it's useful to get an infinite series of zero bytes in various context, like when we're create we want to create a file that has all zero bytes.
We can just read from it and then output it to there, you might say why don't I just make a problem that just right here but this is more general.
So I made a file system here
into food.
But notice before when we wrote this this had a 2627 megabytes. And this was 257.
But now it's only 17.
What happened was as notice it said discarding device blocks.
Done.
It got rid of all the blocks that were, you know, that we're not needed.
So this is pointing out of concept of poles.
So when you talk about the size of file. Normally, we're talking about how many bytes Can you read from it. And this is when you look at the lengthier, this is the logical size.
If you want to read from this file you can read 256 megabytes worth of bytes.
But then there's.
So, then there's the physical spaces how much space it takes one desk. Now you might go. Wait.
Okay, I know blocks.
So.
so let's say you have a file that is 4097 bites long.
It's 4097 bytes long, the logical logical size is 4009 so.
So we have a file that is 4009 seven bites in it that are non zero on a file server 4k blocks, so forth. So this would have so.
So, they won't follow this.
So we have, if we have to. We have a file here, we say it's logical sizes fourth is 4009 sound bites we can read that many non zero bytes from it, it's.
So, that this is the way things normally do.
Yeah.
ls dash LS yeah DS. Yeah, the dash LS exactly it's how you see the physical size.
So, you can't directly, as I said you can't directly see what the actual. Yeah, and the physical size will be in terms of blocks, but the blocks will be traditionally one will by default be one k blocks.
So we have this up.
So let's let's see. So how do we make a file that has. So what we've done right now if we look at this.
So.
Okay so
yeah So Adam you're exactly right so if it was 4095 bites.
Yeah, it's a logical size was 4095 bikes, then exactly it would be
the physical size we 4900 well so the physical size you normally know take physical size in terms of number of blocks,
or at least be divisible if you're talking about number of bytes it's divisible by the block size.
So here remember when we did the stuff I do it.
So.
so this is why, so truncate so nice command.
I'm
so, so Jason, look at this file.
This is one big hole right here.
So, this right here shows you exactly what this is a whole look at this, it's logical size is 10 megabytes.
You know, it's 10 megabytes here it's let's look at that number that's the logical size of it, I can read from it, I can do a
look, you know it's it's all no bites, I can do that for 10 megs what it has no physical space, see this one is saying it takes up four blocks, but this is basically this is saying it takes a 4k, which is usually the smallest size is 4k of anything.
And everything's going to be divisible by four. That's how we know that the physical block sizes 4k.
And here, this is only taking up.
You know, 16 megabytes, even though it's 256 megabytes.
Does that clarify things Jason.
So it's like, so it's not just.
They aren't reserved.
So it's not reserved. There's no space that's been allocated. It's just that this file. if you read from it appears to have 10 megabytes worth of bytes, they're all zeros.
But why do I need to allocate space on disk, just to store zeros. It's basically a, you know.
So where's the file storage, we can only instead of. So, this so what that means is really.
So there are bits in the block us, you know, so, this requires you understanding what happens within I know.
So.
ah, so.
So yeah so you know so this is.
There's no endpoint marker in a file.
Because that will be a special bite pattern.
If you look at the notion of.
So, the way data is stored, it says, How many of.
So it's all in tutorial five cents all the files, the physical size much less a logical size, all these files have a bunch of holes.
If that's true, yes they have holes, but notice when I did the first.
When you did this first DD this did not make holes.
There were no holes that were created by this command. It was the MKFSEXD for that made the holes, because it went through and said, Oh, we don't know this part of the file can be 00, so we can make that whole
exam. Exactly. Now that Jason how you said it's right here, it's a logical sighs what the files, is what the what the file system says it is basically it's how many bytes Can you read from it.
But the physical size is how much space it takes up on disk.
So, so remember when you do these.
You do tutorials. So the way I set this up you notice I'm talking about this tutorial, after you're supposed to have spent time on it.
And notice how this works is that you went through the tutorial. You did stuff that you didn't completely understand. I'm now going through it to try to explain it to you.
And that's how I prefer to do things, because that way you have time to see something and develop questions that I can then explain. If I just show you the clean path through the material.
you don't ever develop questions.
And so you don't build a conceptual model which you really want is like what is going on and I tell you something. Oh, not yet slotted into your memory and see where things fit together so that's how I'm getting these questions is because you tried doing
this and like, wait, what's going on with this what I saw.
Exactly.
So.
So going back to this. So, I think we've gotten this, but but this refers you having a good mental model of this is what is actually happening in the data blocks.
Jason. If you're asking a file system for the next x bites and then you hit the actual physical end before the logical the fossil put into founded by stream.
No, no, that's exactly what will not happen. That is wrong.
You know that that is not at all, does not always all what happened when you have so many logical bites. That's how many bytes you can read. If you read
for the list of data clocks those numbers are those like like it is there's no other little block numbers remember your device, and all the blocks of, you know, a file system max the maps the block the device blocks to file system blocks, it can be a
one to one correspondence or I might say every two physical blocks maps to one file system block. But whatever the cases you have these policies and blocks and these numbers refer to specific, you know, indexes into it's like indexes into an array of
the elements of this array are 4096 bytes.
Okay.
So, these data blocks indexes, it's logically it's a list, but there's lots of ways because you want to be able to account for files.
So if you have a very large file.
Yeah, if your account for very large files. Those may have a list of blocks that can't fit in 4k and whatever, minus whatever metadata you have. So what you'll have to do is have continuation blocks to say, I want to actually no, no blocks in the list
of blocks continues in this block here, and then that will have the list of blocks can continue on this block here.
But it, you know, it varies with the exact format of a file system how this is done.
So, we will talk about file systems using extends all an extent is saying instead of I'm the list of individual blocks, you have a range of blocks and saying, This is the data block you're here.
Now, where does as long as the orders there we don't. As long as you know the logical size you get the last block.
So, you know the logical size you get to the last block and say oh there's only three the first three bites are there, it won't return all the bites, they'll just turned three.
So, in that list of blocks.
If there's a hole in it. For those blocks, either it might be an extent or whatever, you were saying this range.
There's no block so give me you know 50,000 zero blocks.
You don't ever read past the end of its physical size.
Remember, the only thing that can happen is if you come to a whole what happens when you normally the physical size is bigger than the logical size so you can't ever repass that the only time that isn't true is when there's a hole and when you encounter
a hole. It just goes, Oh, you know those next 50 blocks you'd want to read, don't need to read them from disk, they're zeros.
So you can read them really fast because the colonel just mix them up and says, Oh, those are 00 zeros. And just passes an empty buffer back.
So, MKFSMKFSEXD for the. So, he's like you're saying it's a MKFSEXD for was the program that made the holes and MESEXD for is the program from making end for file system.
You'll so yeah so yeah so that's a yeah 50 blocks that are no zeros, it would return 50 times 4096 bytes and then when you get past that it would then start reading from desk.
The reason why we see it yet the part of the reason we zero that the device files to make it easy for it to create it. Otherwise, that would actually have to do things but although I think I'm Chuck, let's do a little test.
So let's do this. I'm going to make it.
Our food, but I'm not going to read from here I will read from Deb, you random, which reads random bytes.
So take a little longer.
So now I've read random bytes.
This is now all there so now.
So we'll do that, starting next week for our.
ls la HSR. Now we see, it's what we expect. And then we do it.
Hi.
So the same thing. It doesn't matter if it was random bytes.
It's just using Dempsey was easy but actually MKFSEXP for zeroed out all the bytes.
So, it didn't matter if they were random or whatever.
So, it's not it's not really about reading fast although we know it certainly, you know, it's nice and fast that you can read something and you don't have to read from disk.
So 10 my file with a five year whole will physically be.
Jason.
Your answer was wrong.
I think you have it basically backwards.
But just remember.
Okay,
a sock your, no, No.
If you guys would get you know yeah you're, you're Miss right off the exact that's that's that's not it so.
So when we talk about the difference between physical and logical physical be bigger, but most by like 4k, because you've just exceeded it by one block,
no physical is normally equal or greater than the logical.
Yeah, a holes are just an optimization. Yeah.
So, basically this physical is greater than large when you just think about this, the number of bytes you can normally read from disk.
You normally expect to use at least that much space on disk to store that many bytes physicals what's on disk logical is know how much it is so want to show you that if you go to the textbook.
Logical is the sum of the physical size, and the holes. Yes, that is still your logical logical is the sum of the physical size of the holes, roughly,
you could have a file that the calculations a little more involved in that because it's all about how much like you couldn't have something where you have a hole of 10 megabytes, and then write two bites.
So, the 10 megabytes or all holes.
But the physical and the physical size is just that.
That goes to bytes, but it's not two bites, it's actually going to be 4k because you have to occupy one block to hold those two bites and not blocked will be using 4k.
So, we use for K for our size.
Because we had two bites. But then we had 10 meg but the actual logical sizes 10 megabytes plus plus two bytes. It's not 10 megabytes plus 4k
going, Okay, I'm going to stop talking about this I spent enough time. Go do experiments and, and try to get clear clarity on this.
But, but of things would you need physical to be greater than or equal to logical.
I'm not gonna go work on it.
But this is all discussed in the textbook also.
Let me see if I can shrink that
whenever
I just want to show so if you go here and file system here.
There we go. So if you go to persistence.
You go to file system implementation.
Show you
what
what I can read PDFs here.
Okay, hold on.
Okay, this is just being stupid.
No, I don't want to read that part of the definitely how close.
I wanted to read.
Okay,
my reader is all of a sudden.
Okay,
try one more time.
We'll try that program again.
This work.
No, this is very much not working.
Sorry this is just annoying.
I tried using, I'm just going to go here and download this year.
Right I use. I use a web browser should night. So if you go here to
this program here.
Go see go to File System implementations so this talks about blocks. So the data region walks through it nicely talks about the I know table, you know, data structure talks about, you know, the metadata talks about directory organization, free space management.
Know.
are you writing
then.
Nope.
Way. Right.
So, if you go here this is the this is an actual implementation of the Berkeley facts file system, which shows you which talks about how disorganized and groups and I nodes and passion The cool thing it has.
Is it, how to deal with large files whereas it show the.
Oh, well that doesn't actually show you the interaction walks, anyway.
But
what I'm saying is, look at that.
If you want to learn more about file systems, it's great stuff in there.
But let's talk talk about the mounting part.
So this tells us information about file system.
So notice what it says here file system you you ID. It says things like the how many I knows does this file system have 65536 So note, can you make more than 6060 64 k files.
No, you can't.
That's how many free I notes are, here's the block size 4096 bytes. The first block starts at zero. So it's just telling you all these things about how big it's a data structure.
It tells you all the stuff about how the data structures put together and.
So now I can mount this file system.
So,
So now we have file system there.
So I'm going to go into seating mount.
Where do.
Okay. So, what I just did there, see a question there.
Just say a cat.
Oh yes, oh yes yes she decided to come in and geography her opening comments here.
Yes.
No. Just say hi to everyone.
Yes, yes oh you're shy.
She is probably a little bit.
Okay.
So, um, so to mod plus T i will explain that a little bit so make one thing to notice there's a couple other bits. I've been told you about directory permissions so normally you have a directory you have read your right you have execute.
So you notice tempt directories, have the plus have that the T thing there.
So there's this thing called the sticky bit that you set on temp directories. When you make fun world rideable then to users could delete each other's files and that gets messy on a shared temp directory, you
The type on like whether they sets the sticky but
if I do have a cat and a dog. Oh yeah. Yeah, they're all.
They're both flying down over there. Actually I have two cats, the other cats not making your parents.
So I'm.
So this, so the reason why I set the sticky bit here was because
now I've got this directory here I want to be able to create files.
So I do.
So first let me do a
pass on.
Delicious two boxes.
So now it is all do.
So it's right now taking up 17 megabytes on disk, but it's logical sizes 256 megabytes.
So let me now try creating
will do random
equals
one.
So I wrote now 10 megabytes here to this of random data.
Now look, the file system logical size state exactly the same here. It's 256 megabytes. But note, the space on disk, change is now taking out 27 megabytes.
Not 17 megabytes because since we've brought into the file system, since we write everything we write in slash Lounge is being stored on the disk.
But the disk that's, yeah it's increased by 10 megabytes, it's being stored in the file system. In this case, stored in the file system it's not a USB stick the backing store for the block device.
What block device is the file now so remember whenever you mount the file system a file system is always mounted from a block device.
So you always say do.
This is a loop back file system.
So, so this what def loop. That's the block device we're using that loop device is like. So if I go into my regular directory.
This is dev mapper LVGO so this there's this thing called Lem which actually provides the storage for the root file system on this virtual machine.
So this is referring to actual.
I can explain lbs later but basically, this corresponds with natural disk, but in a flash mount, this is referring to another device this loop. Whenever you see a loop device.
What that means is there's no storage associated with loop directly, but it can be associated with a file, and that will give the contents of it and that's why it's called loop back because what the system has to do in order to access that data, it goes,
Oh, I want to access something that's in Mount mount is a file system that comes from Blue five, so it goes into Luke five it says Oh, where's my data Luke five oh wait let me go ask, and it goes over to up.
It goes over to the file system which has foo.
So.
masters hammer.
Oh,
we don't have a mouse, but we have a chicken from a neighbor.
That does not come inside the house the cat. The cat and the cat on the chicken have a relationship through a window.
They look at each other.
Have to show you a picture of the chicken.
It's quite funny.
So, this is the loop back. So notice that we read this file and then had to loop it basically as a loop through the stock.
So.
So, this, this, so this this loop back device thing.
So,
this is just a weird thing that do that, but this year.
So, earlier on. So, just to understand what's happening is.
So there's this whole layer of LSTM so if we actually go.
Yay.
so files, LVM to member.
bro.
Oh yeah, that's all we have.
That's all we have here we just have one.
So this is just kind of showing there's another level abstraction here.
My goal like why would you want to do things like this.
I'll show you how homeostasis low.
So like on my physical machine.
So that I have this dev mapper so my data. This is also doing some LVM data, but I'm
so like my, my file says dead SDA one is this 1.8 terabyte file system.
And it's actually encrypted.
So I have to type in a password when I mounted. And that's what part of what LVM lets you do is let you add in the levels of indirection so you get decrypted blocks, so you can take an encrypted file system and turn it into a deep with the process of
book. Yes, well,
okay rushing versus go like classes, over time defeat me.
I have a couple more minutes so just hold on.
So when it says sectors. So when we talk, so.
Okay.
So I have here my system this. So, so I'm quite you know so that's def sta one by I've also does SDI to actually wait a second.
I haven't run cheap party in a while.
So I'm installing it.
Here is. So this is a nice little gooey here for seeing partition information.
And so this is sewing all my devices here. So dev envy me zero is here and I have these ETFs partitions, which are for my booth think oh I've lost 16 megabytes to a Microsoft reserved partition.
Oh no, I forgot I do have windows on this computer.
Oh yeah, I haven't booted that in a long time.
This is G party. yes.
Although otherwise that was a CF disk is that but so my machine was using a little fancier one. So I have three drives on here. I have a, I have this.
This is SDA one, this is my main partition that I use which actually has all my actual data on it that's my home directory and stuff. That's an external drive on this STB is a backup partition, you know basically have an external drive that I am out on
home backup and I drop my backups there.
And then I have this.
This is actually the one that's built this is the envy me Dr that's built into the computer and you see here, I have this envy me one, I have this. All these things are empty Fs.
All these things are empty Fs. This is all windows on this machine, except for this 97 gigabyte partition, which is, which is where am I root partition is.
And that's all, you know, and I've only used 29 gigabytes of that. And so if I can see this if I just typed gf.
You can see the things here. So slash year, I have n. So when we say partitions that means you take a drive and you divide it up into pieces. Those the partitions.
each partition you can say as a range of blocks.
So envy zero and one. I guess that's this the name, this is the name of the drive and as partition five. And you can see partition five because we've got no 1234, and this is the fifth one.
So there's a recovery partition at the end of the drive.
And then that was 650 megabytes and then there's all these, these all these windows stuff and I basically shrunk down the basic data partition. In order to fit in my root partition.
But most of my data is on this external drive. That's a two terabyte drive.
And so you can see my laptop here.
This is what I'm actually running, and it's got this little drive here. This is the external fancy I can see this little thing. This is my external SSD that I use for my, my data and I'm there's another hard drive sitting behind here that I picked up
for backups.
George What a waste a fixed size of blocks. I'm not sure exactly what you're asking.
Well all the devices have fixed size number of boy. Yeah.
So, this is the. Yeah, I'm saying this is how much space, it's made up of blocks. Each block has a fixed size. So if I say information.
So for site information on this it's an ext for partition.
It says that it's this you this, this large it saying the first sector.
Interesting.
So this is what it's saying in terms of sectors.
Does this thing actually have a physical like is the block size here too. Okay.
I'm kind of surprised at saying first sector starts at by 2048 not sure exactly what's going on with that
partition blocks the partition so a partition is just saying, I've got this big disk. I'm not going to treat it as one thing I'm going to split it up.
And so we say this, the range of one to 1 million, you know, you know, is this you know, is this first partition in this range is the next partition this range of the next partition.
And so your file systems are put on a partition. And so the you know there's an offset always at the start of these things. So if I go to a here.
When I look at the information on this partition, you'll see that the first sector. So again here it's time with sector random blocks. But, again, he knows Yo, so this is the first sector starts here, the last one starts here, the total number is this
number.
Sir, it's 56. Oh, thank you. Yeah, I'll stop here. I can answer more questions but this is a deep topic.
There's lots more to see.
But you see you have the tools here to investigate all this stuff.