Operating Systems 2017F Lecture 20 - Revision history

Aleksp: /* Additional Notes */

2017-12-07T17:09:47Z

Additional Notes

← Older revision		Revision as of 17:09, 7 December 2017
Line 176:		Line 176:
	Ls –la /bin/fusermount		Ls –la /bin/fusermount
	Who gives us premissions: just does a normal open and read , the remote kernel enforces these premissions.		Who gives us premissions: just does a normal open and read , the remote kernel enforces these premissions.

			=== Lecture 20 ===

			$ ls -lai less
			* inode numbers are completely different -> they are determined by the filesystem
			* an inode has no meaning outside its fs; it's local to the fs

			What's the filesystem?
			* $ df .
			* $ mount \| grep vda1

			Find fuse fs:
			* $ cat /roc/filesystems \| grep fuse

			When we run "ls" on an sshfs-mounted filesystem
			* ls makes system calls (open, getdents) to the local linux kernel
			* the local kernel sees filesystem is FUSE, calls FUSE routines for open, getdents... (via vfs abstraction)
			* FUSE calls the sshfs process that mounted the filesystem
			:* $ ps aux \| grep sshfs
			* sshfs is itself a process that has to make system calls to talk over the network to the remote system
			* sshfs process sends request to the remote system
			::* via socket system calls
			* remote sshd process receives request (via system calls)
			* remote sshd process accesses local filesystem
			::* makes open, getdents system calls
			::* remote kernel checks vfs, calls ext4 routines to access data
			* remote sshd process responds to request (via system calls)
			* local sshfs process receives response (via system calls)
			* local sshfs process responds to FUSE request
			* FUSE passes data back to vfs layer, then back to requesting process


			* Initial ramdisks - Tue 28 Nov 2017 13:44:34 EST
			:* bootloaded loads both the kernel and initial ramdisk into RAM on boot
			:* /boot -> initrd. = ramdisk


			* $ du -s -h asdf/asdflkj/gkgk
			* $ du -s -h .

			Display a pid's euid
			* $ ps -eo pid, euid \| grep YOUR_PID_HERE
			* $ ps -eo pid, euid, ruid, suid \| grep YOUR_PID_HERE

			List mounts:
			* $ mount -l
			* $ df -aTh

			[https://help.ubuntu.com/community/SSHFS SSHFS]
			[https://www.cs.nmsu.edu/~pfeiffer/fuse-tutorial/ "Writing a FUSE FS: A tutorial"]

HebaJallad at 03:27, 29 November 2017

2017-11-29T03:27:31Z

← Older revision		Revision as of 03:27, 29 November 2017
Line 110:		Line 110:
	** Compare uid, gid of file with uid, gid of process<br>		** Compare uid, gid of file with uid, gid of process<br>
	* In remote system, sftp does file access checks<br>		* In remote system, sftp does file access checks<br>
			********************************************************************
			ADDITIONAL NOTES:

			Fuse: makes you think who is doing what?
			 How it is connected in the kernel .
			 Fuse.sshfs : is what chose the inode numbers?
			o Can they be identical to the other inode numbers?
			 Yes but why?
			 What the process of getting the directories?
			The inodes come from the filesystem which you are accessing

			Who chose the inode number: the filesystem determine the intereprtation of an inode number, Inode is local to a filesystem.
			How is the kernel obtaining information? :
			Fuse : has three kinds
			1) Fuseblk
			2) Fuse
			3) Fusectl
			Summary of sshfs:
			Sshfs process that has to make system call to talk to the network to the remote system.
			Similar to a web request , but the difference separate from that we have a different system calls which is the ls process. To do this, the kernel must call a process.
			Does sshfs create 2 processes? Yes , it actually creates multiple of process
			The remote kernel prespective : file accessing files and sending data back to another system. Remote data is not doing anything special.
			*To verify : ps aux \| grep ssh
			To learn system call sshfs doing ,running local strace as much as you can.
			You can mount a file system which would not allow you to mount any further
			When you update the kernel : the initial ram disk will not be update since it is redundant, it will generate it.
			Can we mount a filesystem in a file?
			*
			What is suspicious? A lot of repetitive inodes
			3rd column : amount of hard links.
			Avoid multiple of binaries if you want a small system, you just need the basics.
			After mounting it it loads the linux system.
			Permissions: the owner can change these permissions, but you can’t take away your own privileges.
			1) Removing a file : unlinking the file, removing a hardlink which modifies the directory in which a hardlink exists. Why do I have the permission to remove it?
			2) Create a file : ouch afile, ls –la afile,
			Premissions:
			1) Other
			2) User
			3) Group
			4) Files system has EUID( very important because processes can run as a user but have permissions of another user ) AND GID: permissions what you have and fs is what ur system sees
			Minix system which manages your CPU.
			Busy Box : One binary to pretend it is different programs and we also have a program which pretends to be many. Give it hard link to different names.
			Soma vs anilclass: anil class is special account. The current directory is owned by soma so how did anilclass was able to create a file? Filesystem understands those primission checks? (prof is not sure about this fact and will check it)
			Map a user ID to your user name? it’s a user space problem, kernel doesn’t care about the username. By looking at the password file : grep 1000 /etc/passwd.
			Where are the permissions done in sshfs? (in the steps
			*sftp search process which will determine the permissions. remote file system and the remote kernel (ext4) will check via sftp. All the file access (grabbing the files)
			Verify : ps aux \| grep sftp
			Who started this sftp: the sshd did.


			Commands:
			Mount \| grep “name”
			Strace ls : directories look the same although they are mounted since ls can not tell , ls is not doing anything differe just opening ur directory and getting data there
			Man sshd : run it and give it aport like 222. Easiest way is using 2 machines on open stack and terminal. It is a good exercise
			Initial Ram disk : to get the real root system to be mounted , similar to memory ll.
			Sudo mount initrd.img-4.10.0-40-generic /mnt : to mount a file in a file system but it didn’t not work and there is an alternative way of doing this
			Anther way : man CPIO : copy files to an form archives . ex : TAR
			CPI - - help : to figure it out
			Ls –a bin \| less
			Cd \ boot : RAM disk
			Du –s –h : in this case (121mb) to see the size of the busy box but the modules takes up half of that space.
			Ls –lai \| head
			Chmod a-w : you are unable to write to this directory anymore
			Chmod o+w : granted permission to other….
			Ls –la /bin/fusermount
			Who gives us premissions: just does a normal open and read , the remote kernel enforces these premissions.

Krithikasaravanan at 02:17, 29 November 2017

2017-11-29T02:17:52Z

← Older revision		Revision as of 02:17, 29 November 2017
Line 103:		Line 103:
	* How to map from user id to username?<br>		* How to map from user id to username?<br>
	** Recall that<br>		** Recall that<br>
	* Kernel knows nothing about usernames<br>		- Kernel knows nothing about usernames<br>
	* This is a userspace process<br>		- This is a userspace process<br>
	* Looks in local password file<br>		- Looks in local password file<br>
	* What determines whether an operation is allowed?<br>		* What determines whether an operation is allowed?<br>
	** Checks to see who owns the file<br>		** Checks to see who owns the file<br>
	** Compare uid, gid of file with uid, gid of process<br>		** Compare uid, gid of file with uid, gid of process<br>
	* In remote system, sftp does file access checks<br>		* In remote system, sftp does file access checks<br>

Krithikasaravanan at 02:17, 29 November 2017

2017-11-29T02:17:08Z

← Older revision		Revision as of 02:17, 29 November 2017
Line 70:		Line 70:
	- compares it with fsuid, fsgid of process <br>		- compares it with fsuid, fsgid of process <br>
	- which is normally same as euid, egid of process <br>		- which is normally same as euid, egid of process <br>
			<br>
			Sshfs<br>
			* Inode values are different in remote vm vs local vm<br>
			* Values increment from 1 in local vm<br>
			* Who's supplying the inode values?<br>
			** Ext4 filesystem in local vm<br>
			** Fuse.sshfs filesystem in remote vm<br>
			* Filesystem determines the interpretation of inode values<br>
			* Inodes have no meaning outside its filesystem<br>
			* Local to its filesystem => hardlinks are different too<br>
			* "strace ls" in both output the same thing<br>
			* Vfs not built into original unix filesystems<br>
			* Try "strace"ing the sshfs call, see how its interacts with fuse<br>
			* Don't need to know exact system calls, but you should know when it has to make system calls and why<br>
			** Needs to access files over the network<br>
			* Understand why you can't do this with regular library calls<br>
			* Removing the root filesystem will trigger a kernel panic<br>
			** Kernel will prevent you from unmounting filesystems that contain other filesystems<br>
			* Kernel modules located in "/lib/modules/"<br>
			* Initial filesystem is loaded into kernel<br>
			** Fake filesystem, throw away after loaded, not persistent<br>
			** Ram disk = filesystem stored in ram<br>
			** Gets the system up to the point where the kernel can load the real filesystem<br>
			** Bootloader has to load both kernel and the initial ram disk into ram<br>
			** Ram disks located in "/boot = initrd.img<...>"<br>
			** Kernel version specific<br>
			** Generated as part of installation of kernel<br>
			* Cpio: copies files to and from archives<br>
			<br>
			* What determines whether you can access a file?<br>
			** The filesystem<br>
			* How to map from user id to username?<br>
			** Recall that<br>
			* Kernel knows nothing about usernames<br>
			* This is a userspace process<br>
			* Looks in local password file<br>
			* What determines whether an operation is allowed?<br>
			** Checks to see who owns the file<br>
			** Compare uid, gid of file with uid, gid of process<br>
			* In remote system, sftp does file access checks<br>

Soma at 20:32, 28 November 2017

2017-11-28T20:32:20Z

← Older revision		Revision as of 20:32, 28 November 2017
Line 1:		Line 1:
			==Video==

			[http://homeostasis.scs.carleton.ca/~soma/os-2017f/lectures/comp3000-2017f-lec20-28Nov2017.mp4 Class Video]

			==Notes==

			===In Class===

			<pre>
			Lecture 20
			----------

			When we run "ls" on an sshfs-mounted filesystem
			* ls makes system calls (open, getdents) to the local Linux kernel
			* the local kernel sees filesystem is FUSE, calls FUSE routines for open, getdents (via vfs abstraction)
			* FUSE calls the sshfs process that mounted the filesystem
			* sshfs process sends request to remote system
			- via socket system calls
			* remote sshd process receives request (via system calls)
			* remote sshd process accesses local filesystem
			- makes open, getdents system calls
			- remote kernel checks vfs, calls ext4 routines to access data
			* remote sshd process responds to request (via system calls)
			* local sshfs process receives response (via system calls)
			* local sshfs process responds to FUSE request
			* FUSE passes data back to vfs layer, then back to requesting process

			Normal file access permission check
			- compare uid, gid of file with uid, gid of process

			But really...
			- compares it with fsuid, fsgid of process
			- which is normally same as euid, egid of process
			</pre>

			===Additional Notes===

	~~== Additional Notes ==~~
	Assignment 4 will be autograded and in fill in the blank form. This will not be like the final exam. <br>		Assignment 4 will be autograded and in fill in the blank form. This will not be like the final exam. <br>
	You should think about who is doing what? Otherwise FUSE won't make much sense <br>		You should think about who is doing what? Otherwise FUSE won't make much sense <br>

Rquaium: /* Additional Notes */

2017-11-28T19:20:08Z

Additional Notes

← Older revision		Revision as of 19:20, 28 November 2017
Line 29:		Line 29:
	This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>		This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>
	To remove a file I need to remove the hardlink from the directory where the hardlink exists <br>		To remove a file I need to remove the hardlink from the directory where the hardlink exists <br>
	The password files maps usernames to user id's (Linux does ~~nto~~ care about your username) <br>		The password files maps usernames to user id's (Linux does not care about your username) <br>
	Normal file access permission check br>		Normal file access permission check br>
	- compare uid, gid of file with uid, gid of process <br>		- compare uid, gid of file with uid, gid of process <br>

Rquaium at 19:17, 28 November 2017

2017-11-28T19:17:55Z

← Older revision		Revision as of 19:17, 28 November 2017
Line 1:		Line 1:

	~~<nowiki>Insert non-formatted text here</nowiki>~~== Additional Notes ==		== Additional Notes ==
	Assignment 4 will be autograded and in fill in the blank form. This will not be like the final exam. <br>		Assignment 4 will be autograded and in fill in the blank form. This will not be like the final exam. <br>
	You should think about who is doing what? Otherwise FUSE won't make much sense <br>		You should think about who is doing what? Otherwise FUSE won't make much sense <br>

Rquaium at 19:16, 28 November 2017

2017-11-28T19:16:02Z

← Older revision		Revision as of 19:16, 28 November 2017
Line 32:		Line 32:
	Normal file access permission check br>		Normal file access permission check br>
	- compare uid, gid of file with uid, gid of process <br>		- compare uid, gid of file with uid, gid of process <br>
	But really <br>		But really: <br>
	- ~~comapres~~ it with fsuid, fs		- compares it with fsuid, fsgid of process <br>
			- which is normally same as euid, egid of process <br>

Rquaium at 19:14, 28 November 2017

2017-11-28T19:14:22Z

← Older revision		Revision as of 19:14, 28 November 2017
Line 10:		Line 10:
	In the SSH there is the filesystem type "fuse.sshfs" <br>		In the SSH there is the filesystem type "fuse.sshfs" <br>
	When you run strace on ls on the two terminals (ssh connection and the local one) you see similar output <br>		When you run strace on ls on the two terminals (ssh connection and the local one) you see similar output <br>
	~~Prof Notes:~~
	When we run "ls" on an ssh-mounted filesystem <br>		When we run "ls" on an ssh-mounted filesystem <br>
	* ls makes system calls (open, getdents) to the local Linux kernel <br>		* ls makes system calls (open, getdents) to the local Linux kernel <br>
Line 25:		Line 24:
	* local sshfs process responds to FUSE request (via system calls)<br>		* local sshfs process responds to FUSE request (via system calls)<br>
	* FUSE passes data back to vfs layer, then back to requesting process <br>		* FUSE passes data back to vfs layer, then back to requesting process <br>
	~~End of prof notes <br>~~
	<br> If you look as lsmod \| less you see a ton of modules. Some of the modules are not necessary to mount the root filesystem.<br>		<br> If you look as lsmod \| less you see a ton of modules. Some of the modules are not necessary to mount the root filesystem.<br>
	ls /lib/modules there are directories that store the kernel modules <br>		ls /lib/modules there are directories that store the kernel modules <br>
Line 31:		Line 29:
	This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>		This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>
	To remove a file I need to remove the hardlink from the directory where the hardlink exists <br>		To remove a file I need to remove the hardlink from the directory where the hardlink exists <br>
			The password files maps usernames to user id's (Linux does nto care about your username) <br>
			Normal file access permission check br>
			- compare uid, gid of file with uid, gid of process <br>
			But really <br>
			- comapres it with fsuid, fs

Rquaium at 19:05, 28 November 2017

2017-11-28T19:05:32Z

← Older revision		Revision as of 19:05, 28 November 2017
Line 30:		Line 30:
	You have to have a filesystem that you mount initially that has a bunch of modules in it. Thus, there is an initial root file system that is necessary to load everything else <br>		You have to have a filesystem that you mount initially that has a bunch of modules in it. Thus, there is an initial root file system that is necessary to load everything else <br>
	This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>		This is the initial RAM disk. This filesystem loads the modules needed for the real filesystem. <br>
			To remove a file I need to remove the hardlink from the directory where the hardlink exists <br>