Operating Systems 2021F: Tutorial 9

From Soma-notes
Jump to navigation Jump to search

Introduction

WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

In this tutorial we will be examining the physical memory mapping of processes with the help of a kernel module that performs a 5-level page table walk for userspace addresses. You may wish to read about 5-level paging.

To get started, we will first examine the source code for 3000physicalview.c and 3000memview2.c, both of which are in 3000physicalview.tar.gz.

Getting Started

  1. Download 3000physicalview.tar.gz and unpack with the command tar xzf 3000physicalview.tar.gz. Compile 3000physicalview and 3000memview2 using the provided Makefile (i.e. by running make).
  2. Insert 3000physicalview by running make insert. Confirm that the module is inserted using lsmod.
  3. Examine the call to copy_from_user and copy_to_user on lines 120 and 132 of 3000physicalview.c. Consider the following:
    • How are these functions different from put_user that we have seen in the previous tutorial?
    • Why are these functions necessary? Couldn't we just access the userspace address directly? What would happen if we did?
  4. 3000physicalview exposes its API to userspace in the form of an ioctl(2) call. Consider the following:
    • What is an ioctl? How is it different from a read or write system call? Hint: check man 2 ioctl.
    • How does 3000physicalview implement its ioctl? What arguments does it take?
    • How does 3000memview2 call the ioctl? What arguments does it pass to the ioctl?

Examining Physical Memory Mappings

  1. With 3000physicalview inserted, run 3000memview2 and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected?
  2. Compare 3000memview2 with 3000memview from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
  3. Do you notice a pattern in the virtual addresses of buf[i]? Is this same pattern present in the physical addresses? Why or why not?
  4. Run 3000memview2 a few more times and consider the following:
    • Are the virtual addresses the same or different between runs? How about physical addresses?
    • Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?

Watching Kernel Memory Allocations

Now let's trace various kernel memory allocations outside of our module.

  1. Run sudo trace -M 100 -K 't:kmem:kmalloc printf "allocated %d bytes at address 0x%llx" args->bytes_alloc, args->ptr' to trace the next 100 slab allocations and print the kernel stack. You may wish to pipe this output into less to read it more easily. What do you notice about the kernel's virtual addresses compared to what you have seen in userspace? Hint: Check the most significant hex digits.
  2. Run sudo trace -M 100 -K 't:kmem:mm_page_alloc printf "allocted 2^%d pages at page frame number %lu" args->order, args->pfn' to trace the next 100 page allocations and print the kernel stack. You may wish to pipe this output into less to read it more easily. Based on what you can see, does page allocation seem to differ from slab allocation? How so?

Code

3000physicalview.tar.gz

3000memview2.c

 1 /* 3000memview2.c  Userland demonstration of 3000physicalview ioctl interface
 2  * Copyright (C) 2020  William Findlay
 3  *
 4  * This program is free software: you can redistribute it and/or modify
 5  * it under the terms of the GNU General Public License as published by
 6  * the Free Software Foundation, either version 3 of the License, or
 7  * (at your option) any later version.
 8  *
 9  * This program is distributed in the hope that it will be useful,
10  * but WITHOUT ANY WARRANTY; without even the implied warranty of
11  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
12  * GNU General Public License for more details.
13  *
14  * You should have received a copy of the GNU General Public License
15  * along with this program.  If not, see <https://www.gnu.org/licenses/>. */
16 
17 #include <stdio.h>
18 #include <stdlib.h>
19 #include <unistd.h>
20 #include <sys/types.h>
21 #include <sys/stat.h>
22 #include <fcntl.h>
23 
24 #define USERSPACE
25 #include "3000physicalview.h"
26 
27 char *gmsg = "Global Message";
28 
29 const int buffer_size = 30;
30 
31 void report_memory(char *prefix, int fd, unsigned long virt)
32 {
33     struct physicalview_memory mem = {};
34     mem.virt = virt;
35 
36     if (ioctl(fd, PHYSICALVIEW_WALK, (unsigned long)&mem))
37     {
38         fprintf(stderr, "Error making ioctl call\n");
39         return;
40     }
41 
42     if (!mem.phys)
43     {
44         printf("%s 0x%016lx -> UNKNOWN\n", prefix, mem.virt);
45         return;
46     }
47 
48     printf("%s 0x%016lx -> 0x%016lx\n", prefix, mem.virt, mem.phys);
49 }
50 
51 int main(int argc, char *argv[], char *envp[])
52 {
53         char format[16];
54         char *lmsg = "Local Message";
55         char *buf[buffer_size];
56         int i;
57 
58         int fd = open("/dev/3000physicalview", O_RDONLY);
59 
60         printf("Memory map report (virtual -> physical)\n");
61         report_memory("argv:      ", fd, (unsigned long)argv);
62         report_memory("argv[0]:   ", fd, (unsigned long)argv[0]);
63         report_memory("envp:      ", fd, (unsigned long)envp);
64         report_memory("envp[0]:   ", fd, (unsigned long)envp[0]);
65 
66         report_memory("lmsg:      ", fd, (unsigned long)lmsg);
67         report_memory("&lmsg:     ", fd, (unsigned long)&lmsg);
68         report_memory("gmsg:      ", fd, (unsigned long)gmsg);
69         report_memory("&gmsg:     ", fd, (unsigned long)&gmsg);
70 
71         report_memory("&buf:      ", fd, (unsigned long)&buf);
72 
73         for (i = 0; i<buffer_size; i++) {
74                 buf[i] = (char *) malloc(4096);
75                 snprintf(format, 16, "buf[%02d]:   ", i);
76                 report_memory(format, fd, (unsigned long)buf[i]);
77         }
78 
79         report_memory("main:      ", fd, (unsigned long)&main);
80         report_memory("puts:      ", fd, (unsigned long)&puts);
81 
82         return 0;
83 }

3000physicalview.c

  1 /* 3000physicalview.c  A kernel module to expose virtual->physical memory mappings in userspace as an ioctl
  2  * Copyright (C) 2020  William Findlay
  3  *
  4  * This program is free software: you can redistribute it and/or modify
  5  * it under the terms of the GNU General Public License as published by
  6  * the Free Software Foundation, either version 3 of the License, or
  7  * (at your option) any later version.
  8  *
  9  * This program is distributed in the hope that it will be useful,
 10  * but WITHOUT ANY WARRANTY; without even the implied warranty of
 11  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 12  * GNU General Public License for more details.
 13  *
 14  * You should have received a copy of the GNU General Public License
 15  * along with this program.  If not, see <https://www.gnu.org/licenses/>. */
 16 
 17 /* Warning: This module is extremely insecure.
 18  * It is designed purely for teaching purposes.
 19  * Using it is stupid unless you are in COMP3000. */
 20 
 21 #include "3000physicalview.h"
 22 
 23 static struct device *device = NULL;
 24 static struct class *class   = NULL;
 25 static int major_number;
 26 
 27 /* Helper functions below this line ---------------- */
 28 
 29 /* Walk the page table of current task for virtual address addr */
 30 static unsigned long get_physical(unsigned long addr)
 31 {
 32     pgd_t *pgd;
 33     p4d_t *p4d;
 34     pud_t *pud;
 35     pmd_t *pmd;
 36     pte_t *pte;
 37 
 38     unsigned long pfn = 0;
 39     unsigned long phys = 0;
 40 
 41     /* Find pgd */
 42     pgd = pgd_offset(current->mm, addr);
 43     if (!pgd || pgd_none(*pgd) || pgd_bad(*pgd))
 44     {
 45         printk(KERN_ERR "Invalid pgd for address 0x%016lx\n", addr);
 46         return phys;
 47     }
 48 
 49     /* Find p4d */
 50     p4d = p4d_offset(pgd, addr);
 51     if (!p4d || p4d_none(*p4d) || p4d_bad(*p4d))
 52     {
 53         printk(KERN_ERR "Invalid p4d for address 0x%016lx\n", addr);
 54         return phys;
 55     }
 56 
 57     /* Find pud */
 58     pud = pud_offset(p4d, addr);
 59     if (!pud || pud_none(*pud) || pud_bad(*pud))
 60     {
 61         printk(KERN_ERR "Invalid pud for address 0x%016lx\n", addr);
 62         return phys;
 63     }
 64 
 65     /* Find pmd */
 66     pmd = pmd_offset(pud, addr);
 67     if (!pmd || pmd_none(*pmd) || pmd_bad(*pmd))
 68     {
 69         printk(KERN_ERR "Invalid pmd for address 0x%016lx\n", addr);
 70         return phys;
 71     }
 72 
 73     /* Find pte */
 74     pte = pte_offset_map(pmd, addr);
 75     if (!pte || pte_none(*pte))
 76     {
 77         printk(KERN_ERR "Invalid pte for address 0x%016lx\n", addr);
 78         return phys;
 79     }
 80 
 81     /* Get physical address of page table entry */
 82     pfn = pte_pfn(*pte);
 83     phys = (pfn << PAGE_SHIFT) + (addr % PAGE_SIZE);
 84 
 85     return phys;
 86 }
 87 
 88 /* Define file operations below this line ------------------- */
 89 
 90 /* Callback to device open */
 91 static int physicalview_open(struct inode * inode, struct file * file)
 92 {
 93     printk(KERN_INFO "3000physicalview open\n");
 94     return 0;
 95 }
 96 
 97 /* Callback to device close */
 98 static int physicalview_release(struct inode * inode, struct file * file)
 99 {
100     printk(KERN_INFO "3000physicalview closed\n");
101     return 0;
102 }
103 
104 /* Callback to device ioctl */
105 static long physicalview_ioctl(struct file *file, unsigned int cmd, unsigned long addr)
106 {
107     struct physicalview_memory *mem;
108     switch (cmd)
109     {
110         case PHYSICALVIEW_WALK:
111             /* Allocate kernel memory for our struct */
112             mem = kmalloc(sizeof(struct physicalview_memory), GFP_KERNEL);
113             if (!mem)
114             {
115                 printk(KERN_ERR "Unable to allocate space for struct\n");
116                 return -EFAULT;
117             }
118 
119             /* Get virt from userspace */
120             if (copy_from_user(mem, (struct physicalview_memory *)addr,
121                         sizeof(struct physicalview_memory)))
122             {
123                 printk(KERN_ERR "Unable to copy struct from user\n");
124                 kfree(mem);
125                 return -EFAULT;
126             }
127 
128             /* Call helper to get physical mapping for virtual address */
129             mem->phys = get_physical(mem->virt);
130 
131             /* Give phys back to userspace */
132             if (copy_to_user((struct physicalview_memory *)addr, mem,
133                         sizeof(struct physicalview_memory)))
134             {
135                 printk(KERN_ERR "Unable to copy struct to user\n");
136                 kfree(mem);
137                 return -EFAULT;
138             }
139 
140             /* Cleanup, cleanup, everybody do their share */
141             kfree(mem);
142             break;
143         default:
144             return -EINVAL;
145     }
146 
147     return 0;
148 }
149 
150 /* Register file operations */
151 static struct file_operations fops = {
152     .open = physicalview_open,
153     .release = physicalview_release,
154     .unlocked_ioctl = physicalview_ioctl,
155 };
156 
157 /* World readable and writable because... security? */
158 static char *physicalview_devnode(struct device *device, umode_t *mode)
159 {
160     if (mode)
161         *mode = 0666;
162     return NULL;
163 }
164 
165 /* Entry and exit points below this line ------------------------------------ */
166 
167 /* Module initialization */
168 int init_module(void)
169 {
170     printk(KERN_INFO "3000physicalview initializing\n");
171 
172     /* Register character device */
173     major_number = register_chrdev(0, DEVICE_NAME, &fops);
174     if (major_number < 0)
175     {
176         goto failed_chrdevreg;
177     }
178 
179     /* Create device class */
180     class = class_create(THIS_MODULE, CLASS_NAME);
181     if (IS_ERR(class))
182     {
183         goto failed_classreg;
184     }
185 
186     /* Set devnode to set our "super secure" permissions from above */
187     class->devnode = physicalview_devnode;
188 
189     /* Create device */
190     device = device_create(class, NULL, MKDEV(major_number, 0), NULL, DEVICE_NAME);
191     if (IS_ERR(device))
192     {
193         goto failed_devreg;
194     }
195 
196     printk(KERN_INFO "3000physicalview is loaded and regstered with major number %d!\n", major_number);
197 
198     return 0;
199 
200     /* NOTREACHED... */
201 
202     /* Errors here */
203 failed_devreg:
204     printk(KERN_ERR "Failed to register 3000physicalview device!\n");
205     class_unregister(class);
206     class_destroy(class);
207 failed_classreg:
208     printk(KERN_ERR "Failed to register 3000physicalview class!\n");
209     unregister_chrdev(major_number, DEVICE_NAME);
210 failed_chrdevreg:
211     printk(KERN_ERR "Failed to register 3000physicalview as a character device!\n");
212     return -1;
213 }
214 
215 
216 /* Module destructor callback */
217 void cleanup_module(void)
218 {
219     /* Cleanup, cleanup, everybody do their share */
220     device_destroy(class, MKDEV(major_number, 0));
221     class_unregister(class);
222     class_destroy(class);
223     unregister_chrdev(major_number, DEVICE_NAME);
224 
225     printk(KERN_INFO "3000physicalview cleanup complete\n");
226 }

3000physicalview.h

 1 /* 3000physicalview.c  A kernel module to expose virtual->physical memory mappings in userspace as an ioctl
 2  * Copyright (C) 2020  William Findlay
 3  *
 4  * This program is free software: you can redistribute it and/or modify
 5  * it under the terms of the GNU General Public License as published by
 6  * the Free Software Foundation, either version 3 of the License, or
 7  * (at your option) any later version.
 8  *
 9  * This program is distributed in the hope that it will be useful,
10  * but WITHOUT ANY WARRANTY; without even the implied warranty of
11  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
12  * GNU General Public License for more details.
13  *
14  * You should have received a copy of the GNU General Public License
15  * along with this program.  If not, see <https://www.gnu.org/licenses/>. */
16 
17 /* Warning: This module is extremely insecure.
18  * It is designed purely for teaching purposes.
19  * Using it is stupid unless you are in COMP3000. */
20 
21 #ifndef PHYSICALVIEW_H
22 #define PHYSICALVIEW_H
23 
24 #ifndef USERSPACE /* Kernelspace only */
25 #include <linux/module.h>
26 #include <linux/init.h>
27 #include <linux/fs.h>
28 #include <linux/device.h>
29 #include <linux/string.h>
30 #include <linux/sched.h>
31 #include <linux/gfp.h>
32 #include <linux/slab.h>
33 #include <linux/mm.h>
34 #include <linux/uaccess.h>
35 #include <asm/uaccess.h>
36 #include <asm/pgtable.h>
37 #include <asm/pgtable_types.h>
38 
39 #define DEVICE_NAME "3000physicalview"
40 #define CLASS_NAME "COMP3000"
41 
42 MODULE_DESCRIPTION("Walk page table for userspace virtual address.");
43 MODULE_AUTHOR("William Findlay");
44 MODULE_LICENSE("GPL");
45 MODULE_VERSION("0.0.1");
46 
47 #else /* Userspace only */
48 #include <sys/ioctl.h>
49 #endif
50 
51 /* Both userspace and kernelspace */
52 
53 struct physicalview_memory
54 {
55     unsigned long virt;
56     unsigned long phys;
57 };
58 
59 /* Ioctl stuff */
60 #define PHYSICALVIEW_BASE 'k'
61 #define PHYSICALVIEW_WALK _IOWR(PHYSICALVIEW_BASE, 1, struct physicalview_memory)
62 
63 #endif /* PHYSICALVIEW_H */
64 </source>
65 
66 ===Makefile===
67 
68 <source lang="makefile" line>
69 obj-m += 3000physicalview.o
70 
71 .PHONY: all, clean, insert, remove
72 
73 all: 3000memview2 3000physicalview.ko
74 
75 3000physicalview.ko: 3000physicalview.c 3000physicalview.h
76 	make ARCH=x86_64 -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
77 
78 3000memview2: 3000memview2.c
79 	gcc -O2 -o 3000memview2 3000memview2.c
80 
81 insert:
82 	sudo insmod 3000physicalview.ko
83 
84 remove:
85 	sudo rmmod 3000physicalview
86 
87 clean:
88 	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
89 	rm 3000memview2