Soma-notes - User contributions [en]

COMP 3000 2012 Week 8 Notes

2012-10-26T13:07:15Z

Spenner2:

Practice questions and their associated answers:

<ol>
<li>
Q: You can create a sparse file in UNIX by writing sequences of zeros to a file. 

A: False. Sparse file means holes. Zeroes aren't holes.
You need to move the file pointer. You have to use a
seek. The data will be READ as zeros, though...
Look up truncate command and strace it. It ain't just writing zeros
</li>
<li>
Q: Are environment variables ``global variables? Specifically, if you change the value of an environment variable X in one process, what happens to the value of X in other processes? 

A:Nothing happens. Env vars aren't globals. They're passed in with Execve. Environments
variables are typically process specific.
</li>
<li>
Q: In UNIX, there are three permissions associated with the user, the user's group, and everyone else. What are those three permissions? 

A: rwx (Read, Write, Execute)
</li>
<li>
Q: If you see a zombie process during normal system operation, how can you get rid of it? 

A: Kill the parent
Why? init will own the zombie process and then kill it
</li>
<li>
Q: I can copy a file using I/O redirection as follows: cat /bin/ls > bar. If I now type ./bar, it won't run. However, if I run another command before typing ./bar, I will get
a file listing. What needed to be changed to get bar to run? 

A: chmod +x
Default non executable. Need to make it executable. (--x)
</li>
<li>
Q: System calls cannot normally be made using standard function calls. What CPU mechanism is used by regular processes to invoke kernel functionality? 

A: Software interrupt. Signals are a form of of software interrupt. 

Note:

"A system call is an entry point into the Linux kernel. Usually, system calls are not invoked directly: instead, most system calls have corresponding C library wrapper functions which perform the steps required (e.g., trapping to kernel mode) in order to invoke the system call. Thus, making a system call looks the same as invoking a normal library function." from man 2 intro 

"The call to the library function itself does not cause a switch to kernel mode (if the execution was not already in kernel mode) and is usually a normal subroutine call (using, for example, a "CALL" assembly instruction in some Instruction set architectures (ISAs)). The actual system call does transfer control to the kernel (and is more implementation-dependent and platform-dependent than the library call abstracting it). For example, in Unix-like systems, "fork" and "execve" are C library functions that in turn execute instructions that invoke the "fork" and "execve" system calls. Making the system call directly in the application code is more complicated and may require embedded assembly code to be used (in C and C++) as well as knowledge of the low-level binary interface for the system call operation, which may be subject to change over time and thus not be part of the application binary interface; the library functions are meant to abstract this away." from http://en.wikipedia.org/wiki/System_call

</li>
<li>

Q: After a fork system call, what runs first, the child process runs or the parent process? 

A: Don't know... Undefined. Can not depend on ordering

</li>
<li>

Q: Signal handlers can be called at almost any point in a process's execution. (T/F) 

A: True. That's the purpose of signals 

Register a normal C function as a signal handler 

Some are implemented by default (ex. sigkill) 

Signals can be called at any point in execution. 

"When a signal is sent, the operating system interrupts the target process's normal flow of execution. Execution can be '''interrupted during any non-atomic instruction.''' If the '''process has previously registered a signal handler, that routine is executed. Otherwise''' the default signal handler is executed." from http://en.wikipedia.org/wiki/Unix_signal

</li>
<li>

Q: System V init scripts run programs sequentially or in parallel? 

A: Sequentially (sysV). Shell script that runs every ''command'' sequentially 

Upstart does it in parallel though. This is the reason sysV is being replaced by Upstart.

</li>
<li>

Q: Without errors, what does the execve system call return? # Who calls a signal handler? 

A: On success, execve() does not return, on error -1 is returned, and errno is set appropriately. 

The program itself calls the signal handler

</li>
<li>

Q: What program is used to check the consistency of the data structures of a filesystem? 

A: fsck <- wrapper function 
fsck.FMT <- called by fsck for a specific format (ex: fsck.ext4)

</li>
<li>

Q: What allocates the storage for environment variables and command-line arguments? 

A: The kernel. The kernel is the only thing that can allocate.(execve) 
This also makes sense because execve is a system call.

</li>
<li>

Q: A file that has a logical size of one gigabyte but only takes up 100K on disk is called what kind of file in UNIX? 

A: Sparse file.

</li>
<li>

Q: What is an easy (but not foolproof way) way to tell that you are running in a virtual environment on Linux? What command do you run to get this information? 

A: lspci, lshw. 
In /proc/, you may be able to also. 
These commands display the machine's "hardware". 

Not /dev/ because that's the abstraction.

</li>
<li>

Q:What is the purpose of the PATH environment variable? 

A: At the command line, the $PATH variable is where the shell looks for binaries/files.
The path is walked when the fullPATH/filename of the command line hasn't been specified

</li>
<li>

Q: When would you expect ltrace to output more lines than strace? When should strace output more than ltrace? 

A: Some libraries make no sys calls (strlen). Libraries that do basic string and math
operations -> no sys calls 

Programs with lots of I/O will use lot of sys calls

</li>
<li>

Q: What command is used to change the priority of a process? 

A: nice and renice 
nicess is policy value (default value and you can modify this) 
priority (dynamic) is based on niceness and the amount of time the process has run on
cpu 
Priority doesn't apply to I/O. I/O will continue despite other processing having high
priority. I/O will always slow down the system.

</li>
<li>

Q: What is the difference between a library call and a function call? 

A: The former requires a library card and a telephone.
 [[User:Spenner2|Spenner2]] ([[User talk:Spenner2|talk]]) nice
</li>
</ol>

COMP 3000 2012 Week 8 Notes

2012-10-26T13:05:34Z

Spenner2:

Practice questions and their associated answers:

<ol>
<li>
Q: You can create a sparse file in UNIX by writing sequences of zeros to a file. 

A: False. Sparse file means holes. Zeroes aren't holes.
You need to move the file pointer. You have to use a
seek. The data will be READ as zeros, though...
Look up truncate command and strace it. It ain't just writing zeros
</li>
<li>
Q: Are environment variables ``global variables? Specifically, if you change the value of an environment variable X in one process, what happens to the value of X in other processes? 

A:Nothing happens. Env vars aren't globals. They're passed in with Execve. Environments
variables are typically process specific.
</li>
<li>
Q: In UNIX, there are three permissions associated with the user, the user's group, and everyone else. What are those three permissions? 

A: rwx (Read, Write, Execute)
</li>
<li>
Q: If you see a zombie process during normal system operation, how can you get rid of it? 

A: Kill the parent
Why? init will own the zombie process and then kill it
</li>
<li>
Q: I can copy a file using I/O redirection as follows: cat /bin/ls > bar. If I now type ./bar, it won't run. However, if I run another command before typing ./bar, I will get
a file listing. What needed to be changed to get bar to run? 

A: chmod +x
Default non executable. Need to make it executable. (--x)
</li>
<li>
Q: System calls cannot normally be made using standard function calls. What CPU mechanism is used by regular processes to invoke kernel functionality? 

A: Software interrupt. Signals are a form of of software interrupt. 

Note:

"A system call is an entry point into the Linux kernel. Usually, system calls are not invoked directly: instead, most system calls have corresponding C library wrapper functions which perform the steps required (e.g., trapping to kernel mode) in order to invoke the system call. Thus, making a system call looks the same as invoking a normal library function." from man 2 intro 

"The call to the library function itself does not cause a switch to kernel mode (if the execution was not already in kernel mode) and is usually a normal subroutine call (using, for example, a "CALL" assembly instruction in some Instruction set architectures (ISAs)). The actual system call does transfer control to the kernel (and is more implementation-dependent and platform-dependent than the library call abstracting it). For example, in Unix-like systems, "fork" and "execve" are C library functions that in turn execute instructions that invoke the "fork" and "execve" system calls. Making the system call directly in the application code is more complicated and may require embedded assembly code to be used (in C and C++) as well as knowledge of the low-level binary interface for the system call operation, which may be subject to change over time and thus not be part of the application binary interface; the library functions are meant to abstract this away." from http://en.wikipedia.org/wiki/System_call

</li>
<li>

Q: After a fork system call, what runs first, the child process runs or the parent process? 

A: Don't know... Undefined. Can not depend on ordering

</li>
<li>

Q: Signal handlers can be called at almost any point in a process's execution. (T/F) 

A: True. That's the purpose of signals 

Register a normal C function as a signal handler 

Some are implemented by default (ex. sigkill) 

Signals can be called at any point in execution. 

"When a signal is sent, the operating system interrupts the target process's normal flow of execution. Execution can be '''interrupted during any non-atomic instruction.''' If the '''process has previously registered a signal handler, that routine is executed. Otherwise''' the default signal handler is executed." from http://en.wikipedia.org/wiki/Unix_signal

</li>
<li>

Q: System V init scripts run programs sequentially or in parallel? 

A: Sequentially (sysV). Shell script that runs every ''command'' sequentially 

Upstart does it in parallel though. This is the reason sysV is being replaced by Upstart.

</li>
<li>

Q: Without errors, what does the execve system call return? # Who calls a signal handler? 

A: execve DOES NOT RETURN ANYTHING. Process image is replaced. 

The program itself calls the signal handler

</li>
<li>

Q: What program is used to check the consistency of the data structures of a filesystem? 

A: fsck <- wrapper function 
fsck.FMT <- called by fsck for a specific format (ex: fsck.ext4)

</li>
<li>

Q: What allocates the storage for environment variables and command-line arguments? 

A: The kernel. The kernel is the only thing that can allocate.(execve) 
This also makes sense because execve is a system call.

</li>
<li>

Q: A file that has a logical size of one gigabyte but only takes up 100K on disk is called what kind of file in UNIX? 

A: Sparse file.

</li>
<li>

Q: What is an easy (but not foolproof way) way to tell that you are running in a virtual environment on Linux? What command do you run to get this information? 

A: lspci, lshw. 
In /proc/, you may be able to also. 
These commands display the machine's "hardware". 

Not /dev/ because that's the abstraction.

</li>
<li>

Q:What is the purpose of the PATH environment variable? 

A: At the command line, the $PATH variable is where the shell looks for binaries/files.
The path is walked when the fullPATH/filename of the command line hasn't been specified

</li>
<li>

Q: When would you expect ltrace to output more lines than strace? When should strace output more than ltrace? 

A: Some libraries make no sys calls (strlen). Libraries that do basic string and math
operations -> no sys calls 

Programs with lots of I/O will use lot of sys calls

</li>
<li>

Q: What command is used to change the priority of a process? 

A: nice and renice 
nicess is policy value (default value and you can modify this) 
priority (dynamic) is based on niceness and the amount of time the process has run on
cpu 
Priority doesn't apply to I/O. I/O will continue despite other processing having high
priority. I/O will always slow down the system.

</li>
<li>

Q: What is the difference between a library call and a function call? 

A: The former requires a library card and a telephone.
 [[User:Spenner2|Spenner2]] ([[User talk:Spenner2|talk]]) nice
</li>
</ol>

COMP 3000 2012 Week 3 Notes

2012-09-30T19:14:16Z

Spenner2: /* Why we're studying unix */

== Shell ==

* useful command line programs
** bc -l
*** command line calculator
** cal
*** command line calendar
** uniq
*** filters out duplicate lines
** sort
*** takes input on STDIN, sorts lines, prints to STDOUT
* vmlinux is the linux kernel with virtual memory {{ref|vmlinux}}
** vmlinuz is the same but compressed
* bash runs on vmlinux
** most programs run on bash
** ls/cd also run on vmlinux
** ls runs as a child process of bash
* to get rid of a misbehaving child
** the parent must kill it
** waits for child to die
** an unreaped child becomes a zombie
** zombies can't be killed, must be reaped
** to kill a zombie, must kill parent
** orphans become a child of init (ward of state)
** init is good at reaping children

== job control ==

* job is a process
** term comes from batch processing
* Useful control sequences
** Ctrl-C
*** terminate job
** Ctrl-Z
*** stop job
* To start a stopped job in the foreground
** fg %n
*** where n is the job number
* To start a stopped job in the background
** bg %n
* To determine jobs that are running
** jobs
* Who manages jobs?
** the shell
** built on kernel functionality

== Why we're studying unix Linux based operating systems==

* all operating systems have roughly the same architecture
* windows is just a variation on unix. Built and designed very similarily.
* "Like a different accent. Same everything, but renamed"
* Slight differences in the way things are done
** Example: On Unix, configs reside in flat text files. In Windows, they live in an opaque key/value store called the registry.

Despite the similarities between OS families, Unix systems tend to be built much more transparently, which makes them a good vehicle to study OS topics

* windows comes from vms
** increment(vms) -> wnt

see [http://www.team.net/mjb/hawg.html Unix - The Hole Hawg].

== Init and Shell Scripts ==

Guest mini-lecture by Ann Fry

* /etc/init.d/
* initialization scripts
* examples
** /etc/init.d/networking [start|stop]
** /etc/init.d/networking [start|stop]
'''Note:''' ''command'' [ ''keyword1'' | ''keyword2'' | ... ] means to execute a command with an optional choice of two or more keywords.
* shell scripts
* start with
** #!/bin/bash
** #!/bin/csh
** #!/bin/ksh

''Enter'' Anil Somayaji.

''[http://en.wikipedia.org/wiki/Exeunt Exeunt]'' Ann ''stage left''.

* Init scripts
* Traditional init scripts run sequentially
* Modern init scripts run in parallel
* executables
* ELF
* A file starting with #!
** next comes the path to an interpreter for the rest of the file
=== Common Directories ===
see [http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard Filesystem Hierarchy Standard] and [http://www.pathname.com/fhs/pub/fhs-2.3.html FHS 2.3]
* /etc
** pronounced ett-see
** system wide configuration files
** different subdirectories and directory structures
** Competing Unix cultures apparent in the naming conventions and directory structures
** /etc/<prog>.conf
** /etc/<prog>/
** /etc/<prog>.d/
* /bin
** system binaries
** often this is local
** Bare minimum of what's needed to boot.
** Important for diskless systems (see /usr/) or think clients.
** Historically, we kept this as small as possible
* /lib
** system libraries
* /home
** user directories
* /usr
** often this is shared
** Historically, the /usr/ directory was a remote file system mounted after the system had fully booted. Servers had more disk space than terminals/thin clients.
** /usr/bin
** user binaries
** /usr/lib
** user libraries

== History of Unix ==

* In the beginning, there was AT&T's Unix
* AT&T gave site licenses with all the source
* Berkeley developed networking (at least TCP/IP) called BSD
** licensed under permissive BSD license
* AT&T forked System V
** Not permissively licensed
* Sun created hardware to run unix and forked SunOS from BSD
* Sun changed from SunOS (BSD) to Solaris (System V)
** people disliked this
* POSIX standard was an outcome of the fragmentation
* Free Software Foundation came out of the fragmentation
** founded by rms ([http://xkcd.com/225/ Richard] M. [http://www.youtube.com/watch?v=1sxNy0qQDkU Stallman])
** not initially a very open development environment
** built all the components of unix called GNU (GNU's Not Unix)
** yacc -> bison
** more -> less
** cc -> gcc
* Anil was sysadmin for a crystalography lab
** SGI IRX boxes
** ran IRIX
* 80386 came out
** capable of running a real unix because of virtual memory
** legal status of BSD kernel became an issue because of lawsuit
** GNU started Hurd
** Linus Torvalds writes Linux kernel
** uses GNU license
** allowed everyone to make contributions
* Eventually BSD legal stuff got figured out
** 386BSD
** FreeBSD
** NetBSD
*** OpenBSD
* Anil's opinion
* Linux has the best driver support of any OS including Windows

== Back to Init ==

* Why do we care?
* Unix is a culture
* the culture shows up all over, like in /etc, /etc/init.d
* Now there are 3 major replacements for system V init scripts
* system V init scripts are just shell scripts run sequentially
* it's desirable to run the initialization in parallel
** oh noes, race conditions
** init scripts have dependencies!!
* init scripts were ordered by convention
** had to figure out all the dependencies
** still not completely figured out
* everything on linux is an open source project
* look around for a good project to document

== References ==

{{note|vmlinux}}: https://en.wikipedia.org/wiki/Vmlinux