Memory Management

Memory Management Lecture 11: Memory Management

Lab 3

Kernel debugging with GDB

In brief, vmlinux is the standard kernel, and bzImage is a compressed version. vmlinux is a statically linked executable file which contains the Linux kernel with executable file formats. When creating a bootable kernel image, the kernel is compressed and created at the location arch/i386/boot/ bzImage. The bzImage (big zImage) format was developed to overcome that problem that the size of the kernels generated by users grew beyond the limits. To decompress the bzImage file and replace the vmlinux, the linux kernel is uptated.

GDB is a source level debugger able to run as remote, the program being debugged is running on a different system than the one running GDB. GDB interacts with the kernel, so the target should NOT be compressed.

File System

The file system is a method of organizing files on a disk. In Linux partition table, /dev/hda represents IDE drive 1; /dev/hdb means IDE drive 2; the rest may be deduced by analogy (The PCs have limited primary partitions with four). The partition in IDE drive is distinguished by number. For instance, /dev/hda5 is the fifth partition in IDE drive 1. For SCSI drive, system uses /dev/sda, /dev/sdb and so on. USB and SATA are also considered as SCSI drive.

Hard Disk update in Linux system (extend file system space)

Scenario: One PC with 10G size hard disk is planed to update with new disk (100G). All things on old one should be moved to the new one, with the same “last modified” on all important files.

Solution: connect the larger disk and copy files from old one to new one.

Detail Steps: Suppose /dev/hda is used by the old one

/dev/hdb1 --> Swap area 2G

/dev/hdb2 --> file system 98G

-Partition Disk:

#fdisk /dev/hdb

display partition table: p

create new partition: n. Also determines the primary partition number, cylinder number and ID 85:

write new partition into disk: w

-Format Disk (to create data structure which is understandable by Linux system )

mke2fs - create an ext2/ext3 filesystem

#mkfs.ext3 /dev/hdb2 --> file system 98G

#mkswap /dev/hdb1 --> swap area 2G

-Mount

There three kinds of information should be known when using mount command

1.File system type of the target to be mounted. Use ‘cat /proc/filesystems’ to get the file type in system and make sure the target is supported.

2.determine the device name (mentioned in part 2 File System)

3.get the mount point. Make sure the directory /mnt is in the system. This one is used as directory containing mount point.

Suppose /mnt/foo exist.

#mount /dev/hdb2 /mnt/foo

note: if system crash, the lost files could be found in the location /mnt/foo/Lost+Found.

Safely Remove: It guarantees the safety of data when the device is stopped, and allows user make the removal process maximum convenient and adjustable. (tell OS not to use it more)

To remove some device, use ‘unmount’ command. For example: #umount /mnt/cdrom.

dd command:

The dd command is one of the original Unix utilities. It can strip header, extract parts of binary files and write into another files; it is used by the Linux kernel Makefiles to make boot image. It can be used to copy and convert magnetic tape formats.

#dd if=<file1> of=<file2> bs=<block size> count=<# block>

note: if=<file1> --> input file, of=<file2> --> output file,

bs=bytes --> block size the optimum value for this operand performance is 4k (1024)

count=# blocks --> copy only this many blocks from the input to the output.

Issue: different size of disk space for file system, how to use the rest part space of the new one?

Solution: when the copy is finished, do resize the new drive to take up the rest of the new drive’s space. #resizee2fs /dev/hdb

cp command:

There are some issues when moving whole bunch of data from old disk to new one.

1.all movable things would be copied into /mnt/foo (the blocked files in Unix still could be copied to new location, but not in windows system)

2.how about to move file system such as /proc or /sys? They are the something just to access the kernel and change status, not the files you want to copy. Waste a lot of space if they are copied.

Solution: cp –a –one_file_sytem /home /mnt/foo

boot loader: It is a small piece of code located at the first block file system.

The bios is the firmware in the ROM of a PC, it is the first program that runs. All other program (for example OS) must be loaded into RAM first. Since the size limitation in bios, when it runs it would load another boot sector, such as LILO (in old Linux system) or GRUB. Their job is simple just to find Linux kernel and load it into memory, and then Linux kernel takes rest part.

LILO: does not know how to find files and relies on a map file to find data blocks of the necessary files.

GRUB: install before using (in this case: grub-install /dev/hdb). It groks file system. Not require to hardcode the block location of kernel location into the boot image, even move it around. GRUB will load correctly.

Format: high level vs. low level

File system is a big data structure. High level format is the process of laying down information needed so that operating system could find around on the disk.

Low level format is just to make disk ready to receive the data; it was used to format floppy disk. It is slow format, including set bits on the disk. Modern operating systems never do low level format on hard disks, USB drives. The quick and regular format are high level format, and the difference is whether to manually erase every block. The low level format is usually to write down information on the disk to organize the internal blocks.

Memory Management

Swap

Swap file is a big file; swap partition is just used for swap file (pre-reserved space). In the space, the files are pre-allocated. Go throughout file interface to access them.

Swapping is a process which can be swapped temporarily out of memory to a backing store, and then brought back into memory for continued execution. The purpose of swapping is for performance. The data on the swap area is contiguous allocation

Now there are several programs running.

The total memory is 512M, Swap file 2G, Disk size 200G

How to do memory allocation, keep the memory program needed? Just keep two programs running and kick off one program? No, it only needs to kick out partial pieces of program, which means do not need all the programs in the memory to run. There are a lot of pages, is it necessary to read all pages at all time? No! some pages are read at once; some staff of the kernel window access the means of memory, but others do not. So kick the others out. It is possible to save us a good 200M. Similarly, if Azuiens is running at the background, the file transfer all the codes you add, why do they have to be in the memory? Kick it out. For word, you hate the grammar check and turn it off. Have they stayed at the memory? No. Just keep the program you need in the memory.

How much the program should be loaded into memory to run? At least one page (4K). We can do everything in 4K (swap), but why we buy additional memory for PC? Swap in 4K causing the speed slow down (at disk speed). You may want to keep speed, all things in the memory, so how much is enough? The process of loading is kind of slow (even today system to load 200M) and why need to load? The designer of modern memory system is said to be laze, only load what you need to load. For Firefox, you just need to load the part making it executable.

Now how does the operating system know how much is loaded? So there is an optimal strategy. It makes the operating system predict what memory should be loaded. Load and make it ready before executing. This is optimal strategy for memory management, only keep things necessary based on the features. But predict future is very hard.

How can you do predict? You know the past, summarize the past, the future will be like past. Modern operating system assumes anything you access will be used again. Every time you access some pieces of new memory, some new data and some new codes loaded from disk which is so slow. But if you use them multiple times, you get good performance.

To implement it, use Least Recently Used (LRU) Algorithm: if we use the recent past as an approximation of the near future, then we will replace the page that has not been used for the longest period time.

The implementation:

- Every page entry has a counter; every time page is referenced through this entry, copy the clock into the counter

- When a page needs to be changed, look at the counters to determine which are to change.

Swap has different organization; pages here could be quickly accessed. When the grogram is running, some new pages are needed and some are kicked out of the memory. But you need to know some certain duplication here. Firefox is executable, and some piece of codes is kicked out of the memory. Do we need to write to the swap? No, we do not, because it already on the disk. All needs to write to the swap is the parts you changed.

When do u write it into swap file? Think of two strategies.

- One is to be aggressive. Keep doing this at any time and kicking something out, while it is already in the swap file, and we do not need to wait them to write to disk. Because we have to write to disk to free memory, it will take your time.

- The other one is lazily to do it. Only do write when you kick something out. The OS has a strategy to keep enough free memory at any time while the program wants more memory whether OS can give to it.

Page Fault

When you load many big programs, and things just slow down. Maybe you would wonder that I do not really run these things, they should not do anything. Because they still occupy some resource, they are busy reading and writing swap file. Keep generating page fault. (If there is ever a reference to a page, first reference will trap to OS Þ page fault). It is important to keep the page fault rate low in page swap system. Otherwise, the effective access time increases, slowing process execution dramatically.