Operating Systems 2017F Lecture 23 - Revision history

HebaJallad: /* In Class */

2017-12-12T04:20:02Z

In Class

@@ Line 76: / Line 76: @@
 -10
 </pre>
 ===Additional Notes===

Aleksp: /* Additional Notes */

2017-12-09T16:16:33Z

Additional Notes

← Older revision		Revision as of 16:16, 9 December 2017
Line 130:		Line 130:
	* what system calls are made (and not made)? <br>		* what system calls are made (and not made)? <br>
	* short sequence of system calls? 6-10 calls <br>		* short sequence of system calls? 6-10 calls <br>

			Lecture 23

			How can you tell a process has been compromised (i.e. from outside the process)?
			* The process is working on behalf of an attacker
			<br>
			Classic way to do this:
			* Pattern matching -> signatures
			::* is the process running bad code?
			::* is the process doing bad things?
			:::* i.e. /etc/passwd -> password prog. should only be able to access it
			:::* if a process is going to do bad things, it's going to make "bad" system calls
			<br>
			So, how can we tell if a process is making bad system calls?
			<br>
			Don't want to sit and write complex rules to determine:
			* Which programs should make which system calls, etc.
			::* i.e. policy based systems and sandboxing of processes
			<br>
			Therefore, we want the computer to determine what call is good/bad.
			* i.e. use machine learning
			<br>
			The issue is, we have to demonstrate, not just "good", but also "bad"<br>
			We have lots of examples of "bad", but is not necessarily representative of "bad"<br>
			Difficult to enumerate all possible occurences of "bad"<br>
			<br>
			However, we know how systems "normally" behave
			<br>
			How about teaching the system to differentiate normal from abnormal?
			* Assume:
			::* normal is "good"
			::* abnormal may be bad
			:::* there is no guarantee that abnormal is bad, however, if it's bad, but not abnormal... we're in trouble
			:::* false positives are bad (i.e. abnormal but not good)
			<br>
			How do we detect abnormal system calls?
			* a machine learning problem
			* the system should learn as it runs and decide when it has learned "enough"
			::* learn normal patterns of system calls over time
			:::* once learned enough, watch for abnormal system calls
			<br>
			Could we do the learning within processes?
			* possible, but all data would have to come from the kernel
			<br>
			Want something fast and simple, so it can be implemented in the kernel
			* you're right at ground-level, where decisions are being made
			::* i.e. if bad system call being made -> can stop it immediatelly

			* don't want to be training a neural network to do this -> too complicated, too much overhead
			<br>
			''Thu 7 Dec 2017 13:53:01 EST -> Video of observing system calls, ls vs. xclock''
			<br>
			=== First assumption: ===
			ignore the arguments system calls are making -> look at the calls themselves
			* but, different processes invoke different calls -> how to compare them?
			* even multi-threaded processes will mirror the structure of the code in the calls it makes

			=== Second assumption: ===
			look at the ordering of system calls on a per-thread, per-process basis
			* doesn't make sense to think of 'ls' system calls in the context of 'xclock' system calls
			<br>
			Therefore, any profiling will be based on the code being executed

			=== Third assumption: ===
			characterize processes based on the executable they are running
			* model per executable, with each one trained on multiple processes
			<br>
			How do we model the trace of system calls coming from a process?
			* How often do different system calls hapen? -> frequency analysis
			* high variance -> the calls change frequently
			::* i.e. ls of a large dir vs. small dir

			::* What system calls does a process makes or doesn't make?

			::* Rather than examining if a process does or doesn't make a particular system call, instead look at short sequences of system calls being made.
			:::* What is the variation in the pattern of sequences of calls being made? A compromised program will be detectable.

			:::* Table lookup of sequences made by a program and compare against new sequences
			<br>
			How short is a short sequence of system calls? -> 6 to 10

			When a program is running, the short sequences define the control flow path of the program<br>
			The short sequences together represent the control flow<br>
			When a program is exploited, an abnormal control flow, an uncommon path, is being used<br>
			<br>
			Try the simple hack first, rather than designing/engineering a complex solution
			* the simple hack will often present valuable insights

Soma at 02:16, 8 December 2017

2017-12-08T02:16:17Z

@@ Line 1: / Line 1: @@
-== Additional Notes ==
+[http://homeostasis.scs.carleton.ca/~soma/os-2017f/lectures/comp3000-2017f-lec23-07Dec2017.mp4 Lecture 23 Video]
 Written solutions for midterm exam are on the course webpage <br>
 Assignment 4 <br>

Rquaium: /* Additional Notes */

2017-12-07T19:10:57Z

Additional Notes

← Older revision		Revision as of 19:10, 7 December 2017
Line 24:		Line 24:
	I can't teach good versus bad if I don't know what is bad <br>		I can't teach good versus bad if I don't know what is bad <br>
	But I know how systems behave normally <br>		But I know how systems behave normally <br>
	How about teaching the system ~~tod ifferentiate~~ normal from abnormal <br>		How about teaching the system to differentiate normal from abnormal <br>
	- normal is good <br>		- normal is good <br>
	- abnormal may be bad <br>		- abnormal may be bad <br>
Line 46:		Line 46:
	Third Assumption: characterize processes based on the executable they are running <br>		Third Assumption: characterize processes based on the executable they are running <br>
	model per executable, each trained on multiple processes <br>		model per executable, each trained on multiple processes <br>
			How to model the trace of system calls coming from a procesd <br>
			* frequency analysis? <br>
			- on a system call basis <br>
			- high variance <br>
			* what system calls are made (and not made)? <br>
			* short sequence of system calls? 6-10 calls <br>

Rquaium: /* Additional Notes */

2017-12-07T18:58:52Z

Additional Notes

← Older revision		Revision as of 18:58, 7 December 2017
Line 21:		Line 21:
	- and they don't work well either <br>		- and they don't work well either <br>
	Make the computer solve this problem <br>		Make the computer solve this problem <br>
			- Use Machine Learning <br>
			I can't teach good versus bad if I don't know what is bad <br>
			But I know how systems behave normally <br>
			How about teaching the system tod ifferentiate normal from abnormal <br>
			- normal is good <br>
			- abnormal may be bad <br>
			abnormal but not bad => false positive <br>
			False positives can be a big issue because they may cause ppl to not trust the machine's detection capabilities <br>
			<br>
			How can we detect abnormal system calls? <br>
			Learn normal patterns of system calls over time <br>
			Once you've learned enough, watch for abnormal system calls <br>
			Since I'm lazy, I want to learn it as it runs <br>
			- and automatically decide when it has learned enough <br>
			<br>
			Could I do the learning in a process (or set of processes)? <br>
			you could, but all data would have to come from the kernel <br>
			Want something fast and simple <br>
			- implement in the kernel <br>
			How simple could it be? <br>
			<br>
			First assumption: ignore arguments <br>
			Second assumption: look at ordering of system calls on a per-thread, per-process basis <br>
			Third Assumption: characterize processes based on the executable they are running <br>
			model per executable, each trained on multiple processes <br>

Rquaium: /* Additional Notes */

2017-12-07T18:39:56Z

Additional Notes

← Older revision		Revision as of 18:39, 7 December 2017
Line 14:		Line 14:
	- is it running "bad code" <br>		- is it running "bad code" <br>
	- is it doing "bad things" <br>		- is it doing "bad things" <br>
	-- For example: password program start modifying files other than etc/password. You ~~should~~ specify rules to prevent this. <br>		-- For example: password program start modifying files other than etc/password. You could specify rules to prevent this. <br>
	-- bad system calls		-- bad system calls<br>
			For a process to do damage, it has to make "bad" system calls <br>
			I want to be lazy <br>
			- complex rules are a pain <br>
			- and they don't work well either <br>
			Make the computer solve this problem <br>

Rquaium: /* Additional Notes */

2017-12-07T18:34:48Z

Additional Notes

← Older revision		Revision as of 18:34, 7 December 2017
Line 5:		Line 5:
	Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>		Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>
	Q4: Both lines because first you start from 1 and then increment from there. <br>		Q4: Both lines because first you start from 1 and then increment from there. <br>
	Q11: Only bs because it is write. Would be ibs and bs if it was read.		Q11: Only bs because it is write. Would be ibs and bs if it was read. <br>
	Q12: Local kernel forwards the write system call but doesn't actually make the system call. Kernels don't make system calls.		Q12: Local kernel forwards the write system call but doesn't actually make the system call. Kernels don't make system calls.<br>
			Lecture 23 Prof Notes <br>
			-----------------
			How can you tell when a process has been compromised. <br>
			- from outside the process <br>
			Use signatures <br>
			- is it running "bad code" <br>
			- is it doing "bad things" <br>
			-- For example: password program start modifying files other than etc/password. You should specify rules to prevent this. <br>
			-- bad system calls

Rquaium: /* Additional Notes */

2017-12-07T18:29:34Z

Additional Notes

← Older revision		Revision as of 18:29, 7 December 2017
Line 5:		Line 5:
	Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>		Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>
	Q4: Both lines because first you start from 1 and then increment from there. <br>		Q4: Both lines because first you start from 1 and then increment from there. <br>
	Q12:		Q11: Only bs because it is write. Would be ibs and bs if it was read.
			Q12: Local kernel forwards the write system call but doesn't actually make the system call. Kernels don't make system calls.

Rquaium: /* Additional Notes */

2017-12-07T18:20:54Z

Additional Notes

← Older revision		Revision as of 18:20, 7 December 2017
Line 5:		Line 5:
	Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>		Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>
	Q4: Both lines because first you start from 1 and then increment from there. <br>		Q4: Both lines because first you start from 1 and then increment from there. <br>
			Q12:

Rquaium: /* Additional Notes */

2017-12-07T18:16:24Z

Additional Notes

← Older revision		Revision as of 18:16, 7 December 2017
Line 1:		Line 1:

	== Additional Notes ==		== Additional Notes ==
	~~Solutions~~ for midterm exam are on the course webpage <br>		Written solutions for midterm exam are on the course webpage <br>
			Assignment 4 <br>
			Q2: SSH keygen generates the secret key and the public key file. The private key is stored in the private key file: .ssh/id_rsa <br>
			Q4: Both lines because first you start from 1 and then increment from there. <br>