Okay, let's get started here.
Okay Good morning everyone.
Let's get started. Let's see updates.
So interviews.
So first off on interviews.
I'm not sure exactly I just need to figure out my schedule for next week and then I'll open up slots for next week. I've got a couple of things that seemed quite locked down so once that's done, I will know my schedule and I'll put up the slots and I
will post a message for that.
For those of you who have been.
So,
three.
yes, Yes.
So,
so one thing to note in the chat. If I do not respond to your comments. If it's a.
If it's a jokey comment I might ignore it. If it's a serious comment, and I have I haven't done, I might have missed it in my scroll past, and I haven't gone through it so please, so I'm
so much, you know, I'm
Dr. The assignments we do the, the 19th, so I'll be there so you'll have a week. And actually, as you'll see, everything in the assignment is from the tutorials for a little bit.
But if you guys I mean if there's some other deadlines I should know about that a little, I don't have much flexibility the last assignment I have a little bit of flexibility for this assignment.
Unless there's any particular questions I want to kind of get started here and let's talk about up
a file systems.
So I'm.
Say I want to talk about file systems, and we started talking about file systems, I think, you know, we were talking about file systems talking about I nodes and stuff.
I don't think I gone into quite,
quite the depth I want to do.
So to recall what we are to.
So file systems.
So it's a.
So, you're having more think of a file system, you know file system, you think it was a collection of files and directories that's kind of the most intuitive way.
You could also think of it as, as a namespace for I know numbers.
You know, it's also because a way to trans, form a block device into place.
store files.
was the data structure that you access the file API.
Oh, file descriptors good question, file descriptors will come up here in this in a second.
So will remind me if I don't talk about that later but that should come up here so.
So there's lots of different ways you can think of file systems here, but here's a more.
So from the Colonel's perspective when you think about a file system.
What you should.
When you mount a file system on a directory the item number start from one again.
They don't necessarily start from if What if it's a.
The new block device you created, typically will start one it doesn't have to start one but certainly if you mount a new device as a completely new set of I nodes.
Yes. And if it's a brand new file system yes flash will typically have the I note of one but that's file system specific as I will upload it. So, but from the crowds perspective.
So, from the curls perspective.
It has a file system, a system called interface for accessing files.
So I think the best way to think about this is more of a functional kind of sense is that the colonel gets a request for a file operation of some kind of a directory or whatever, and as part of it is the open you know open door you have this path name.
It goes like.
Who knows how to handle that and what it does it parses that name, and then figures out okay for that containing directory. What file system is that part of, and whatever file system it's part of it has a bunch of file system registered for different
parts of the file hierarchy, and then goes, Hey, you do the open. Hey, you do the read Hey you do the right.
So, there's an entire abstraction layer for different file systems. And that's known as the virtual file system layer.
If you look at
looking slash proc file,
file systems.
So, so this is, this is on my machine, let me see if I am
still not quite no there's still a lot. So you see, there's
all these different things up.
So,
so this is what so so basically these are the different ones you see there's all these ones that are no Dev, What they mean is.
So, remember files fundamentally are an API for the Linux kernel.
And that API can ground out in storage, that's a on an SSD or even on a network file system or something, versus ones that are kinda different.
So, yeah, so.
So this is, so this is what is so I'm trying to get at what the file system is directly know cutters so I now I'm on my machine here.
I look at these file systems here, I can say df here.
No.
Okay.
Yeah okay so if I just say def.
Okay so, so, so we're going to file descriptors in a little bit.
So does each is each directory its own file system. That's one question I got every file, these are not their own.
Every directory is not its own file system. So, in particular, let me show you what I'm talking about here so if I go to my desktop here, and I'm going to, I can go df.so.
Okay so so you know we'll get to in a second here. So, so right now we're in, so deep right now we are a mountain so it's mounted on slash.
This is the file system this is the device that's that's there for this. So, basically the storage for slash is in dev mapper LG are there and we look at flash la mapper
that.
So you can see this is a block device.
That's corresponds to, you know, that's where the files for slash store. Now, this is by far not the only file system that's there.
And if I do df dash a.
All those virtual file system so there's no Corresponding Source let's. So this is all the, all the file systems here so these are all the mount points for where these are so if you go into any of these directories, and we do a df will get something different.
So like if I go into slash sis.
I'll see that this is sis Fs that's mounted here I go into slack proc.
This is proc Fs, which is something else. Now we already know about what pocket fest is, this is the thing that gives us interface to the colonel to find out say what processes are running are the things like this tells us all kinds of information about
the currently running Colonel, but there's no storage associated with it. This is just access to Colonel data structures, that's generated on the fly as needed.
There's no, you know, so like it literally when I look at you know for one eight or nine, I'm looking at a data structure in the kernel, and we're going to learn more about those data structures, but basically, all this is either directly.
You're really running a function which just gathers stuff from random strokes, and then turns them into strings and put some here and and that's what you get when you do a read from these.
So I I'm getting I'm getting to the open files in a second so.
So, we're so you. So these are all things here that are generated automatically and we can tell that things are a little funny here, because, well, here the I know they're kind of weird, and notice also that the size of all these files is zero.
accrue a data structures in the kernel we will see them shortly but they're just see strong point, they're just see structure, it's. That's it, with lots of pointers and things and, you know, now, so virtual file systems, was if I said well it's up the
term
pseudo file systems of the CEO.
I should change that. Okay.
which
file systems.
So, So why would we have.
So, so can file systems include block devices and co files up
file systems you have, use a real file systems and pseudo file system pseudo file systems have no corresponding storage real file systems are stored on a block device.
So, here, notice like these files are like if I cat version, version clear very clearly has a length.
Yet here it says version has size zero.
So, that's kind of messed up. But again, it's not actually being a real fast, but now. What about the real file systems.
What am I mean by real estate I mean things that store stuff.
So if I want to find out more.
I can just take mount.
Yeah, real Fs will file systems on block devices okay so.
So other significant differences between real file systems. Oh yes. So let's look at.
So any file says, No, a file system can be mounted pseudo file systems can be mounted in pseudo philosophy mounted just as much as a real file sitting on it.
But when you mount a pseudo file system, you're not getting access to any new storage you're getting access to some new capability.
So, we may be clear about that so.
So like if you look in slash right.
So slasher go for example.
So yeah so basically. So,
Basically, yes.
So I'm just giving you some example of some things are slash run is just a temporary file system, which is used for state for currently running processes.
The idea here is that if a process just needs like what is the current problem was looking for and see what's their story.
It's not a very thinking up like a megabyte, but we can see now.
Lots of you seen lots of PID files.
This is kind of a nice thing here
where it's a temporary file system. What do I mean, is that amount of after Monica. Know what I mean is, it's a temporary file system and of course when you walk, you know,
No. So this is just so it is this is true of temp Fs temp Fs. So here.
So, in this last run slash run is the mount point for the file system, and temp Fs is the file system type.
And so in here. If I were to. So, let me do a demonstration.
So now pseudo.
I run.
You see, now I have a file called important
so important is there.
So I've got this file here now, I'm going to reboot.
And I've now rebooted that virtual machine.
Let's see here in a second here.
So, know what just happened.
Data is lost when sin.
So notice I just created this file.
I put it in slash run.
It was there, it seemed perfectly fine. But when I rebooted the system, the data went, poof.
And that's not a bug. That's a feature, because everything in slash run should go poof, when you reboot, because it's all data that only makes sense, while the system is currently rebooted.
Once I reboot the system, and all that is gonna be cut. That's all.
No, current data. And so older systems didn't have temp Fs so much, so you'd have files sitting there in temp Fs.
And that it would just go away.
Yes. Yeah, so everything is last run is stored in temp of us.
But, no, it's good to
notice.
So Tim is not tempt Fs.
But you will notice.
So, so the other stuff and so take a look and slash run every program just sticks that their sticks up there. But the idea is stuff that should automatically go away it's for things like process IDs locks and things but there's also a tab chapters not
attempt that fast but it's also temporary.
So what some other ones. So note the.
So no, so this is what you're seeing
in your VM.
Such tape only gets erased when you reboot.
That's all the time it gets rebooted
erase with a script when it's a it's a logging in and out doesn't cause it to reboot, your kids, your race.
Yes.
So question is why is this template template fast.
Okay.
I was wondering like what would that was
up.
Okay lock files we will we will talk about lock files at us so
okay that's that's another concept to get through.
Type EXTEXT for, let's So, remember we're taught the class file system.
So these are all the file system, there's all these know Deb ones is all these things but notice that there's things like x two x two x force vi fat.
Butter Fs arm.
current ones are.
XFSUXTXTX for.
Oh,
What is it called.
So, there is lots of file system types. That's the thing to understand your squash Fs is.
I'll get to squash a fast.
falls into all those were originally designed for it.
Yeah, there's there's basically Why are there so many file system solve it's just know you make a new system you make your own file system you go like it's got, I mean, they all have different, different performance characteristics.
One key thing to know is, some are designed for pauses compliant system politics is this big standardized thing for Unix systems, others are not in particular that goes into.
Yeah.
Yeah, magnetic yeah right so
that's good.
That's good way to characterize it.
So.
So what does this all mean when I say file systems here.
stubbornness i'm saying is what my stubbornness is not necessarily those are really good design reason people just want to make their own.
Oh,
there's actually a big one, especially an open source world what what cut what license the code is under the proprietary things certain kind of some open source licenses are compatible with each other and so people have to reinvent things.
So, one thing to mainly know is that ext four is really the one that is currently standardized.
That's a most systems work.
Thanks.
Yes, creating a pen to squash Fs file systems.
So, a read only file system is useful when you want a collection of files. So while.
So,
so there's these.
If you're looking at a CD old CDs, DVDs, blu ray discs optical media.
Those things are normally read only. So there's no, there are various features of file systems that are designed to facilitate reading and writing of data.
If you only have to support read operations, you can optimize the data structure in different ways. Basically you put a lot of effort to organize things initially to make reads as fast as possible.
And it turns out this is particularly important for optical media, because how things are laid out on this can have great impact and how fast Davis read.
So you couldn't know so that's why you want to really find some but also for
is protecting the blue racy was read only a good way to protect the desk.
The idea is that a blu ray CD, you a blu ray disc, you can write to it is not physically possible um you can get reliable blu rays but the medium itself.
Know isn't designed to be rideable so you make the file system itself is read only few miles will make the way you organize the data we don't leave Why put a read write file system on a read only media it's just like, It's a waste.
Know what I'm saying no.
You have to have a way of making DVDs, it turns out, you press them.
It's basically you, you make the file system and then you basically make a mold and you press it. That's. Look up, look up the creation optical Miss, that's what well burners are a bit different.
I actually had there's a whole.
Yeah, if you look there they have all kinds of things on optical media.
Yeah, lava lamps was fun.
They'll tell you about what it means when you actually burn a disk and stuff. I'm not going to talk about now because frankly that's so obsolete now, so.
So it's not even worth me explaining too much but there's like a time to read only things but it's often used up. Installing the system.
So, there are times when you want read only file systems.
Okay, lovely lady the thread. Oh, by the way, at 1111, I just, we should.
I can set alarm for 1111.
Because I will turn off my camera at 1111 for one minute.
Yeah.
So I'll. So that basically at 1111 I'll just turn off the camera and mute myself for one minute, a moment of silence, you know, a minute of silence and then I'll come back.
Today is Remembrance Day Yes, sorry I do not have puppies since I'm in the US. I don't know where my puppies have gone, and I do not have a Tim Hortons or something nearby where I can pick up one.
Anyway, I'm glad to remember that. I thought that for class anyway so look optical media.
The US does not do poppies, the US does not recognize Remembrance Day.
They don't, Because remember, this is really a world war one thing in the US really wasn't as involved in World War One.
So, yeah, it's very common well.
So
what's so let's
make a file system.
So, from a file system.
Make.
Oh that's so.
So here's the one weird thing about
this tutorial, which are, which is a little funny so first off let's make a file.
Oh.
Okay.
Yes, making a swap file is making a swap files kind of similar so what I've done first is, First we're going to.
So what is it counting So, yes.
Count time block sizes.
So.
so can we not just have us touch.
no.
We could not just us touch.
touch this.
Yeah okay so yeah we'll, we'll get to truncate in a second here. So,
we've talked about files, and how they're stored.
So, I've done now is I've created this file here called.
Let's not what it says right here.
So you got fake.
So, so you got fake blocks here is now there.
And if we look, if we less fake blocks.
You notice.
So we're reading from an infinite set of zero bytes.
Now what I'm going to do is make a file system in there.
So then we have this whole message here, where it's made a file system so now there's a lot of things we're talking about the structure of the file system.
So the question is can we kept dev No, so let's try.
It's gonna be a size zero.
Just like touch.
Yeah, so that's the problem.
So, the key thing is we want a file of a fixed size. So now we've done is we've created a data structure in there. It's now we can look at this file. Look at this thing.
Now the. This is what I was trying to remember.
I don't think it'll work actually for this.
I guess it just read part of it.
But that's not very useful.
Uh, let's.
Okay.
So if we do.
Why do display.
So,
so if you do.
So we can do.
So if you look here.
Okay, so now let's do the first.
So.
So you notice here, what, what od does it says, here's what they are we got a no no no no no no no. So those are all no bites, and it has this many times, so it's like it just repeats.
That's all there is, there is nothing else.
But if we do this to this.
Now look, we got stuff here.
And notice we've got these, so there's faces here you see like from here to here, it says this is all repeated knows, but there is stuff
in it, there are characters that we wrote.
Okay.
So,
these are so
you missed the part with didn't know these characters. So this is your need to know what the characters are. This is a.
Let's do it again.
So let's, so fake box to.
We just do this sort of with I think
it's just no bites. That's all it is. It's empty things, because we got it from dev zero. Remember, you can read an infinite number of zeros. From dev zero.
Joe copy dev zero.
We don't want so if you just try to copy it was it was copied zero.
What's happening now.
Exactly.
So I just took up my to put against.
Don't. So, Deb zero is an infinite number of zero bytes.
Yeah, it can be. But yeah, I see Unix gives you lots of power to really hurt yourself. This is why I say right now, very very important run backups on your come 3000 on your VM, when you're doing these, these things, especially for tutorial five and six,
you're doing dangerous stuff. You are messing with the way you log into the system and then you're messing with storage. One wrong command and you erase everything when you get hurt other systems because that's why you're in your virtual machine, but
you can blow everything away so run your backup scripts.
So, is dev zero actually infinite.
Yes, up.
What is dev zero.
What is definitely use, we just used it.
It's a character device.
Exactly, so, so, So there is no
runs code for the character device.
So, remember the API.
What's actually happening with dev zero. You're doing a read. When you do a read, you have a, you just give it a buffer right we've now seen read system calls.
So you give the, the kernel of buffer saying here, fill this next so many bites, instead of going to a device and trying to figure out what the storage is this says, oh, I've got a buffer zeros.
Here you go.
So.
So, what it's it's it's it's all yeah it's infinite, because it's just me it's just a function. All you're doing is literally when you call dev zero you're calling, read, and then it says, okay, zeros.
There's no storage corresponding to it.
Yeah, Exactly. Well one return no but it just fills up the buffer that you give it and it never gives end to file, so you can, as many reads as you do have it, you'll get that many buffers of zero so if you're going to keep me.
You want to keep reading for it for terabytes of things you'll get terabytes of zeros. Now, there are other ones like this.
So you could essentially recreate dev zero, using a script. I'm
kind of.
We're going to talk about how you implement character devices but you would have to the trick is that you can access it using the file API.
So, it's not a complex thing and.
But you have to be able to implement the interface, and there's, there's a lot more there but just trying to get your head around this abstraction.
That's a
beyond the scope of this course, if I have time I can talk about randomness but that is so far beyond this course, that's really crypto course they talk about random this.
So if we ever fake blocks files. So now if I try mounting.
So let's try mounting.
So I look here and I've mounted it, I've now got something in Mount Greg Good job, if I go into slash mount.
It says, sis.
So, we'll talk about what that saying now here in a second, but let's now.
Get accept that so
CDs now.
So we still have our thing there
was true,
That we got this here, let's.
I'm going to go. So, What do we just do here, fake box to.
Okay, let's try and make it fit so first of all, I just try it so let me try the from fake.
So I'm going to run this command.
It is enough.
Oh, crap.
Okay, next.
Okay that's good to know.
Okay.
We're.
Thanks everyone.
Let me continue with where we were,
let's step back for a second here so.
So,
so to create initialize district, we have MKMS think of this just like initializing a hash table.
Do it setting up the initial buckets, you're creating, you know, you're just laying things down so you can navigate setting up the pointers, whatever.
But when you talk about file systems. We also talk about validating and repairing the data structure.
Because now we don't normally think about data structures as needing repairs and and needing to be done because.
So, if you think of like stuff in memory in your house it was like if they get messed up you just restart the program.
So, file systems are designed to be recoverable when they're when they're damaged and that's, that's a mindset you have to have in mind when you're thinking about file systems is yes their data structures, but their data structures that are designed to
be long live because they're storing all of your data, and they're designed to last a cost reboots and other things.
Cosmic rays is a much more often thing then
it's become more and more an issue as data densities have become more things and so in fact actually modern file systems have a lot of mechanisms to deal with that, not so much ext for but things, but other ones and I think file systems are moving more
towards having higher data integrity things but when I say software I mean any software.
I'm saying any software, anything in your computer.
Any bugging your computer could cause your, your, you know, something to get corrupted in your storage and it's one thing you reboot the computer, but what is some straight bits get written or something.
How can you know the glitches out.
Oh, that's funny. Okay, Mario 64 glitches. So, FSCK.
So FSK tries his best to repair things up. Just
one question digital file systems have FCKO make there.
Yeah. No, definitely not.
Yeah, so there are runtime things so if you understand why FSA K is useful for real file systems, and not useful for pseudo file systems.
You understand something important.
does windows have. Yeah.
Oh, what's the windows equivalent.
Check this Yeah, there's a guy you all you guys didn't use older versions of Windows.
We'll, we'll talk file system checkers are interesting problem. But if we.
Oh, I'm on this machine. I got myself confused. I thought I wasn't on it.
So will the yeah so, yeah.
So, you need his own specialized repair mechanisms. So if we do ever ck.
You'll notice that FSCK is really a rapper that tries to run the correct FFC K, depending upon the file system type.
So like if we if it's a AXT for file system it'll run EXFSEKEXT for, or they'll run, you know, elephants and eat you know butter Fs one it'll run, you know BTR know or Fs or the fat or MS DOS or whatever.
So there's all these, can you create your own file system check type, you can, that means you have to make your own file system, which you can, but it's not so simple.
So you'd have to expect you know extend things and I'll be talking about all the places that you have to implement it. So, but
we're not going to make file system in this course. Well, we're going to what we're making possible not going to make a new file system type. We just we're just making one up.
So, file systems have lots of redundancy and how data is stored in order to facilitate this, so.
Okay, so please in a file system, always bidirectional so you always refer to one another and then there's also.
So,
do you think of the super block like the root note of the tree it's really the start of the data structure. So everything else refers to it as also other metadata everything about it so if you lose a super block.
You're screwed.
So, when you make a file system most file systems store multiple copies of the superblock.
block is a fixed size of storage. The summer blocks.
Yes, like so. Yes, so bidirectional so it's like a.
So, it's a data structure, instead of pointers, we have blocked number references.
So the aren't addresses it's which block you're talking about and so we always do.
We have not talked at all about physical versus logical addresses. That's really a concept from virtual memory so physical, put it aside to a first of abstract, you could say these block numbers are.
It just this thing of his block numbers.
We talked about physical versus virtual Oh, but anyway, we are, we are out of time here.
Clearly, there is more to the story I will keep going. Next week, but if concepts here are feeling fuzzy that is perfectly fine. That is what tutorials five and six, and assignment, three are designed to help you with.
So, work on the tutorials will keep talking about this next week.