COMP 3000 Lab 2 2011 - Revision history

Saran: /* Part B */

2011-10-05T15:56:29Z

Part B

← Older revision		Revision as of 15:56, 5 October 2011
Line 143:		Line 143:

	==Part B==		==Part B==
			Part B is marked by Saran. Please email him for any queries.
	# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.		# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.
	# Guest OSes do not require the same device drivers as the host OS, as they do not have access to real hardware directly. Virtualization softwares use [http://en.wikipedia.org/wiki/Device_driver#Virtual_device_drivers Virtual Device Drivers] to emulate hardware for guest OSes.		# Guest OSes do not require the same device drivers as the host OS, as they do not have access to real hardware directly. Virtualization softwares use [http://en.wikipedia.org/wiki/Device_driver#Virtual_device_drivers Virtual Device Drivers] to emulate hardware for guest OSes.

Saran: /* Part A */

2011-10-05T15:56:03Z

Part A

← Older revision		Revision as of 15:56, 5 October 2011
Line 108:		Line 108:
	=Answers=		=Answers=
	==Part A==		==Part A==
			Part A is marked by Ann. Please email her for any queries.
	#		#
	## (2 points - 1 mark for each o/s tested) Loading up a virtual machine of Ubuntu 11.04 within virtualbox or vmware should take approximately 2 minutes and 27 seconds.		## (2 points - 1 mark for each o/s tested) Loading up a virtual machine of Ubuntu 11.04 within virtualbox or vmware should take approximately 2 minutes and 27 seconds.
Line 139:		Line 140:
	return 0;		return 0;
	}		}
	}		}


	==Part B==		==Part B==

Saran: /* Part B */

2011-10-04T23:54:56Z

Part B

← Older revision		Revision as of 23:54, 4 October 2011
Line 197:		Line 197:
	millis2 = tv.tv_usec; */		millis2 = tv.tv_usec; */
	//printf("difference secs = %ld, millis = %ld\n",secs2-secs1,millis2-millis1);		//printf("difference secs = %ld, millis = %ld\n",secs2-secs1,millis2-millis1);

	fclose(a);		fclose(a);
	return 0;		return 0;
	}		}

Saran: /* Part B */

2011-10-04T23:54:23Z

Part B

← Older revision		Revision as of 23:54, 4 October 2011
Line 157:		Line 157:

	<br> The objective of this question is to write a program that is relatively less efficient on a guest. You will receive full marks if your code shows that you've understood both the points mentioned above.		<br> The objective of this question is to write a program that is relatively less efficient on a guest. You will receive full marks if your code shows that you've understood both the points mentioned above.

			<br> Update - For some of you, time measurement using clock() seems to be erroneous in Virtual Machines. You can instead use timestamp as shown in the commented code below.

	<br> I used the code shown below :		<br> I used the code shown below :
Line 162:		Line 164:
	#include <stdio.h>		#include <stdio.h>
	#include <time.h>		#include <time.h>
			//#include<sys/time.h>

	#define TIMEVARS() clock_t start, stop; double sec		#define TIMEVARS() clock_t start, stop; double sec
Line 173:		Line 176:
	int n1 = 12345,n2 = 10987654;		int n1 = 12345,n2 = 10987654;
	int i= 0;		int i= 0;
			//static struct timeval tv;
			//long int secs1, millis1,secs2,millis2;
	FILE *a;		FILE *a;
	a=fopen("tmp0001","w");		a=fopen("tmp0001","w");
	TIMEVARS();		TIMEVARS();
	TIMESTART("Starting work");		TIMESTART("Starting work");
			/*
			gettimeofday(&tv,NULL);
			secs1 = tv.tv_sec;
			millis1 = tv.tv_usec; */
	for(i = n1; i < n2; i++) {		for(i = n1; i < n2; i++) {
	fprintf(a,"%d ",i);		fprintf(a,"%d ",i);
Line 183:		Line 192:
	}		}
	TIMESTOP();		TIMESTOP();
			/*
			gettimeofday(&tv,NULL);
			secs2 = tv.tv_sec;
			millis2 = tv.tv_usec; */
			//printf("difference secs = %ld, millis = %ld\n",secs2-secs1,millis2-millis1);

	fclose(a);		fclose(a);
	return 0;		return 0;
	}		}

Saran: /* Part B */

2011-10-03T22:00:01Z

Part B

← Older revision		Revision as of 22:00, 3 October 2011
Line 144:		Line 144:
	==Part B==		==Part B==
	# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.		# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.
	# Guest OSes do not require the same device drivers as the host OS, as they do not have access to ~~read~~ hardware directly. Virtualization ~~software~~ use [http://en.wikipedia.org/wiki/Device_driver#Virtual_device_drivers Virtual Device Drivers] to emulate hardware for guest OSes.		# Guest OSes do not require the same device drivers as the host OS, as they do not have access to real hardware directly. Virtualization softwares use [http://en.wikipedia.org/wiki/Device_driver#Virtual_device_drivers Virtual Device Drivers] to emulate hardware for guest OSes.
	# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.		# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.
	# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.		# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.

Saran: /* Part B */

2011-10-03T21:41:27Z

Part B

← Older revision		Revision as of 21:41, 3 October 2011
Line 144:		Line 144:
	==Part B==		==Part B==
	# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.		# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.
	# Guest OSes do not require the same device drivers as the host OS, as they do not have access to read hardware directly. Virtualization software use Virtual Device Drivers to emulate hardware for guest OSes ~~(link)~~.		# Guest OSes do not require the same device drivers as the host OS, as they do not have access to read hardware directly. Virtualization software use [http://en.wikipedia.org/wiki/Device_driver#Virtual_device_drivers Virtual Device Drivers] to emulate hardware for guest OSes.
	# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.		# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.
	# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.		# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.

Saran: /* Part B */

2011-10-03T21:32:36Z

Part B

← Older revision		Revision as of 21:32, 3 October 2011
Line 147:		Line 147:
	# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.		# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.
	# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.		# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.
	# Programs that involve just computation, i.e memory and the CPU, run at near native efficiency. Unless there's a software emulation of the whole processor, a guest OS is just like a process on the host machine with preallocated memory and CPU. ~~Shadow page tables (~~ http://lwn.net/Articles/216794/ ) are used to make memory accesses fast.		# Programs that involve just computation, i.e memory and the CPU, run at near native efficiency. Unless there's a software emulation of the whole processor, a guest OS is just like a process on the host machine with preallocated memory and CPU. [http://lwn.net/Articles/216794/ Shadow Page Tables] are used to make memory accesses fast.
	# Programs on the Guest OS that use ethernet cards, accelerated graphics or hard disks directly run slowly because these devices are emulated by the host. A guest OS does not have access to real hardware. Techniques developed recently like IOMMU virtualization seek to overcome this problem.		# Programs on the Guest OS that use ethernet cards, accelerated graphics or hard disks directly run slowly because these devices are emulated by the host. A guest OS does not have access to real hardware. Techniques developed recently like IOMMU virtualization seek to overcome this problem.
	# This question asks for two strategies employed by Virtual machine software ( a hypervisor ) to improve a guest's performance. One could be to modify the guest OS drivers to do I/O more efficiently ( for ex. avoiding the IDE/SATA interfaces ). A second method is to modify the CPU to reduce the cost of switching in and out of the hypervisor, i.e hardware virtualization. If you answered some methods for application performance optimization, you'll be graded based on that, owing to the lack of clarity of the question.		# This question asks for two strategies employed by Virtual machine software ( a hypervisor ) to improve a guest's performance. One could be to modify the guest OS drivers to do I/O more efficiently ( for ex. avoiding the IDE/SATA interfaces ). A second method is to modify the CPU to reduce the cost of switching in and out of the hypervisor, i.e hardware virtualization. If you answered some methods for application performance optimization, you'll be graded based on that, owing to the lack of clarity of the question.

Saran: /* Part B */

2011-10-03T21:30:25Z

Part B

← Older revision		Revision as of 21:30, 3 October 2011
Line 154:		Line 154:
	# The same host/guest on the same hypervisor, and writing to the same partition on the hard disk, it took 16.080 seconds to write an 84 MB file on the host machine, and 22.560 seconds on the guest machine - a slowdown of about 40%. Your answers may vary but you should see a slowdown. A few things to note here :		# The same host/guest on the same hypervisor, and writing to the same partition on the hard disk, it took 16.080 seconds to write an 84 MB file on the host machine, and 22.560 seconds on the guest machine - a slowdown of about 40%. Your answers may vary but you should see a slowdown. A few things to note here :
	## We don't want the CPU to take up time calculating stuff enabling the disk drive to write contents. We want to keep the drive busy at all times. You should disable the "modulo" operation if you're using the above code.		## We don't want the CPU to take up time calculating stuff enabling the disk drive to write contents. We want to keep the drive busy at all times. You should disable the "modulo" operation if you're using the above code.
	## fprintf() writes to a buffer, not to the underlying media. fflush() has to be used to flush contents from the userspace. Further, the kernel may keep its own buffer. Use fsync() to ensure that it gets written to the disk. Without flushing the stream completely, both Host and Guest times were similar at approximately 1.2 seconds, which just goes to show you how much buffering takes place.		## fprintf() writes to a buffer, not to the underlying stream/media directly. fflush() has to be used to flush contents from the userspace. Further, the kernel may keep its own buffer. Use fsync() to ensure that it gets written to the disk. Without flushing the stream completely, both Host and Guest times were similar at approximately 1.2 seconds, which just goes to show you how much buffering takes place.

	<br> The objective of this question is to write a program that is relatively less efficient on a guest. You will receive full marks if your code shows that you've understood both the points mentioned above.		<br> The objective of this question is to write a program that is relatively less efficient on a guest. You will receive full marks if your code shows that you've understood both the points mentioned above.

Saran at 21:28, 3 October 2011

2011-10-03T21:28:47Z

← Older revision		Revision as of 21:28, 3 October 2011
Line 151:		Line 151:
	# This question asks for two strategies employed by Virtual machine software ( a hypervisor ) to improve a guest's performance. One could be to modify the guest OS drivers to do I/O more efficiently ( for ex. avoiding the IDE/SATA interfaces ). A second method is to modify the CPU to reduce the cost of switching in and out of the hypervisor, i.e hardware virtualization. If you answered some methods for application performance optimization, you'll be graded based on that, owing to the lack of clarity of the question.		# This question asks for two strategies employed by Virtual machine software ( a hypervisor ) to improve a guest's performance. One could be to modify the guest OS drivers to do I/O more efficiently ( for ex. avoiding the IDE/SATA interfaces ). A second method is to modify the CPU to reduce the cost of switching in and out of the hypervisor, i.e hardware virtualization. If you answered some methods for application performance optimization, you'll be graded based on that, owing to the lack of clarity of the question.
	# Guest Additions for VMs comprise device drivers and system applications installed on Guest OS to optimize their performance (ex. better Video support ) and usability ( ex. shared clipboard )		# Guest Additions for VMs comprise device drivers and system applications installed on Guest OS to optimize their performance (ex. better Video support ) and usability ( ex. shared clipboard )
			# I used Ubuntu 11.04 as host, and Arch as Guest on Qemu/KVM. Hardware is Core i7 with 4 cores, and 6 GB RAM. I chose n1 = 12345, n2 = 258765. It took 10.660 seconds to run on the host, and 10.670 seconds on the guest which is approx slower by an insignificant 0.1% (Your answers may vary). Simultaneously, the run times were similar because of a multi core machine.
			# The same host/guest on the same hypervisor, and writing to the same partition on the hard disk, it took 16.080 seconds to write an 84 MB file on the host machine, and 22.560 seconds on the guest machine - a slowdown of about 40%. Your answers may vary but you should see a slowdown. A few things to note here :
			## We don't want the CPU to take up time calculating stuff enabling the disk drive to write contents. We want to keep the drive busy at all times. You should disable the "modulo" operation if you're using the above code.
			## fprintf() writes to a buffer, not to the underlying media. fflush() has to be used to flush contents from the userspace. Further, the kernel may keep its own buffer. Use fsync() to ensure that it gets written to the disk. Without flushing the stream completely, both Host and Guest times were similar at approximately 1.2 seconds, which just goes to show you how much buffering takes place.

			<br> The objective of this question is to write a program that is relatively less efficient on a guest. You will receive full marks if your code shows that you've understood both the points mentioned above.

			<br> I used the code shown below :
			/* io_perf.c for linux */
			#include <stdio.h>
			#include <time.h>

			#define TIMEVARS() clock_t start, stop; double sec
			#define TIMESTART(msg) printf(msg "..."); fflush(stdout); start = clock()
			#define TIMESTOP() stop = clock(); \
			sec = ((double)(stop-start))/CLOCKS_PER_SEC; \
			printf("done (%.3f seconds)\n", sec)

			int main()
			{
			int n1 = 12345,n2 = 10987654;
			int i= 0;
			FILE *a;
			a=fopen("tmp0001","w");
			TIMEVARS();
			TIMESTART("Starting work");
			for(i = n1; i < n2; i++) {
			fprintf(a,"%d ",i);
			fflush(a);
			fsync(a);
			}
			TIMESTOP();
			fclose(a);
			return 0;
			}

Saran at 17:50, 3 October 2011

2011-10-03T17:50:31Z

← Older revision		Revision as of 17:50, 3 October 2011
Line 105:		Line 105:

	}		}

			=Answers=
			==Part A==
			#
			## (2 points - 1 mark for each o/s tested) Loading up a virtual machine of Ubuntu 11.04 within virtualbox or vmware should take approximately 2 minutes and 27 seconds.
			## (2 points - 1 mark for each o/s tested) This answer depends on which distributions you have chosen. For Arch, freebsd: you are given a barebones distribution without a gui. For Fedora: similar gui background, For Knoppix: Textual based interface with audio readout.
			# (1 mark) - provide username
			# (1 mark) - CPU name should only vary slightly, provide proof of this. (yes)
			# (1 mark) - video is dependant on the virtual machine software within the virtual machine & should be specified as such. (no)
			# (1 mark) - libc_start_main() puts()
			# (1 mark) - compiled statically will not call dynamic functions
			# (2 marks - 1 for static list, 1 for dynamic list) - static - execve, uname, brk, brk, set_thread_area, brk, brk, fstat64, mmap2, write, exit_group (total number: 8 distinct functions) - dynamic - execve, brk, access, open, fstat64, mmap2, close, access, open, read, fstat64, mmap2, mprotect, mmap2, mmap2, close, mmap2, set_thread_area, mprotect, mprotect, mprotect, munmap, fstat64, mmap2, write, exit_group (total number: 13 distinct functions)
			# (1 mark) - ./hello-dyn > hello-output or ./hello-static > hello-output
			# (1 mark) - Program should include something similar to this:

			/* hellotofile.c */
			#include <stdio.h>
			#include <unistd.h>
			FILE *out;
			char filename[] = "hello-output";

			int main(int argc, char *argv[])
			{
			out = fopen(filename,"w");
			if (out == NULL)
			{
			fprintf(stderr, "Couldn't open output file %s!\n", filename);
			return 1;
			}
			else
			{
			fprintf(out, "Hello, world!\n");
			return 0;
			}
			}


			==Part B==
			# Guest OS's hard disk is just a file on the host operating system's file system. For example, VHD (Virtual Hard Disk), and VMDK (Virtual Machine Disk) are some popular file formats.
			# Guest OSes do not require the same device drivers as the host OS, as they do not have access to read hardware directly. Virtualization software use Virtual Device Drivers to emulate hardware for guest OSes (link).
			# open() takes as input a file path and returns a file descriptor which can later be used to perform read() or write(). This is a stateful method, because the kernel has to keep track of every file descriptor's underlying I/O stream for every process. An alternative would be to have stateless (like HTTP) system calls for read() and write() which directly take file paths and data offset as input instead of file descriptors obviating the need for the kernel to store metadata for file accesses. The former method places the burden of maintaining state on the kernel, and the latter method places the burden of maintaining state, if needed, on the user.
			# Dynamic library calls involve the execution of code from within the users process in the unprivileged user mode. System calls involve user/kernel mode switching and are executed by the privileged ring 0. However, note that standard dynamic libraries provide wrappers for system calls and it is generally a good idea to use those instead of system calls directly.
			# Programs that involve just computation, i.e memory and the CPU, run at near native efficiency. Unless there's a software emulation of the whole processor, a guest OS is just like a process on the host machine with preallocated memory and CPU. Shadow page tables ( http://lwn.net/Articles/216794/ ) are used to make memory accesses fast.
			# Programs on the Guest OS that use ethernet cards, accelerated graphics or hard disks directly run slowly because these devices are emulated by the host. A guest OS does not have access to real hardware. Techniques developed recently like IOMMU virtualization seek to overcome this problem.
			# This question asks for two strategies employed by Virtual machine software ( a hypervisor ) to improve a guest's performance. One could be to modify the guest OS drivers to do I/O more efficiently ( for ex. avoiding the IDE/SATA interfaces ). A second method is to modify the CPU to reduce the cost of switching in and out of the hypervisor, i.e hardware virtualization. If you answered some methods for application performance optimization, you'll be graded based on that, owing to the lack of clarity of the question.
			# Guest Additions for VMs comprise device drivers and system applications installed on Guest OS to optimize their performance (ex. better Video support ) and usability ( ex. shared clipboard )