Soma: /* Watching Kernel Memory Allocations */

2022-11-07T16:26:06Z

Watching Kernel Memory Allocations

← Older revision		Revision as of 16:26, 7 November 2022
Line 32:		Line 32:
	==Watching Kernel Memory Allocations==		==Watching Kernel Memory Allocations==
	Now let's trace various kernel memory allocations outside of our module.		Now let's trace various kernel memory allocations outside of our module.
	# ~~Run~~ <tt>~~sudo~~ trace -M 100 -K 't:kmem:kmalloc printf "allocated %d bytes at address 0x%llx" args->bytes_alloc, args->ptr'</tt> to trace the next 100 slab allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. What do you notice about the kernel's virtual addresses compared to what you have seen in userspace? Hint: Check the most significant hex digits.		# As root run <tt>trace -M 100 -K 't:kmem:kmalloc printf "allocated %d bytes at address 0x%llx" args->bytes_alloc, args->ptr'</tt> to trace the next 100 slab allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. What do you notice about the kernel's virtual addresses compared to what you have seen in userspace? Hint: Check the most significant hex digits.
	# ~~Run~~ <tt>~~sudo~~ trace -M 100 -K 't:kmem:mm_page_alloc printf "allocted 2^%d pages at page frame number %lu" args->order, args->pfn'</tt> to trace the next 100 page allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. Based on what you can see, does page allocation seem to differ from slab allocation? How so?		# As root run <tt>trace -M 100 -K 't:kmem:mm_page_alloc printf "allocted 2^%d pages at page frame number %lu" args->order, args->pfn'</tt> to trace the next 100 page allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. Based on what you can see, does page allocation seem to differ from slab allocation? How so?

	==Code==		==Code==

Soma: Created page with "==Introduction== WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine. In this tutorial we will be examining the physical memory mapping of processes with the help of a kernel module that performs a 5-..."

2022-11-07T16:25:21Z

Created page with "==Introduction== WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine. In this tutorial we will be examining the physical memory mapping of processes with the help of a kernel module that performs a 5-..."

New page

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and
may result in crashes or loss of data. Additionally, questions may not work as expected on
a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

In this tutorial we will be examining the physical memory mapping
of processes with the help of a kernel module that performs a 5-level
page table walk for userspace addresses. You may wish to
[https://en.m.wikipedia.org/wiki/Intel_5-level_paging read about 5-level paging].

To get started, we will first examine the source code for <tt>3000physicalview.c</tt> and <tt>3000memview2.c</tt>, both of which are in [https://homeostasis.scs.carleton.ca/~soma/os-2021f/code/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Getting Started==
# Download [https://homeostasis.scs.carleton.ca/~soma/os-2021f/code/3000physicalview.tar.gz 3000physicalview.tar.gz] and unpack with the command <tt>tar xzf 3000physicalview.tar.gz</tt>. Compile <tt>3000physicalview</tt> and <tt>3000memview2</tt> using the provided Makefile (i.e. by running <tt>make</tt>).
# Insert <tt>3000physicalview</tt> by running <tt>make insert</tt>. Confirm that the module is inserted using <tt>lsmod</tt>.
# Examine the call to <tt>copy_from_user</tt> and <tt>copy_to_user</tt> on lines 120 and 132 of <tt>3000physicalview.c</tt>. Consider the following:
#* How are these functions different from <tt>put_user</tt> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What would happen if we did?
# <tt>3000physicalview</tt> exposes its API to userspace in the form of an <tt>ioctl(2)</tt> call. Consider the following:
#* What is an <tt>ioctl</tt>? How is it different from a <tt>read</tt> or <tt>write</tt> system call? Hint: check <tt>man 2 ioctl</tt>.
#* How does <tt>3000physicalview</tt> implement its <tt>ioctl</tt>? What arguments does it take?
#* How does <tt>3000memview2</tt> call the <tt>ioctl</tt>? What arguments does it pass to the <tt>ioctl</tt>?

==Examining Physical Memory Mappings==
# With <tt>3000physicalview</tt> inserted, run <tt>3000memview2</tt> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected?
# Compare <tt>3000memview2</tt> with <tt>3000memview</tt> from [[Operating Systems 2021F: Tutorial 2|Tutorial 2]]. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <tt>buf[i]</tt>? Is this same pattern present in the physical addresses? Why or why not?
# Run <tt>3000memview2</tt> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?

==Watching Kernel Memory Allocations==
Now let's trace various kernel memory allocations outside of our module.
# Run <tt>sudo trace -M 100 -K 't:kmem:kmalloc printf "allocated %d bytes at address 0x%llx" args->bytes_alloc, args->ptr'</tt> to trace the next 100 slab allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. What do you notice about the kernel's virtual addresses compared to what you have seen in userspace? Hint: Check the most significant hex digits.
# Run <tt>sudo trace -M 100 -K 't:kmem:mm_page_alloc printf "allocted 2^%d pages at page frame number %lu" args->order, args->pfn'</tt> to trace the next 100 page allocations and print the kernel stack. You may wish to pipe this output into <tt>less</tt> to read it more easily. Based on what you can see, does page allocation seem to differ from slab allocation? How so?

==Code==

[https://homeostasis.scs.carleton.ca/~soma/os-2021f/code/3000physicalview.tar.gz 3000physicalview.tar.gz]

===3000memview2.c===

<syntaxhighlight lang="c" line>
/* 3000memview2.c Userland demonstration of 3000physicalview ioctl interface
* Copyright (C) 2020 William Findlay
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>. */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

#define USERSPACE
#include "3000physicalview.h"

char *gmsg = "Global Message";

const int buffer_size = 30;

void report_memory(char *prefix, int fd, unsigned long virt)
{
struct physicalview_memory mem = {};
mem.virt = virt;

if (ioctl(fd, PHYSICALVIEW_WALK, (unsigned long)&mem))
{
fprintf(stderr, "Error making ioctl call\n");
return;
}

if (!mem.phys)
{
printf("%s 0x%016lx -> UNKNOWN\n", prefix, mem.virt);
return;
}

printf("%s 0x%016lx -> 0x%016lx\n", prefix, mem.virt, mem.phys);
}

int main(int argc, char *argv[], char *envp[])
{
char format[16];
char *lmsg = "Local Message";
char *buf[buffer_size];
int i;

int fd = open("/dev/3000physicalview", O_RDONLY);

printf("Memory map report (virtual -> physical)\n");
report_memory("argv: ", fd, (unsigned long)argv);
report_memory("argv[0]: ", fd, (unsigned long)argv[0]);
report_memory("envp: ", fd, (unsigned long)envp);
report_memory("envp[0]: ", fd, (unsigned long)envp[0]);

report_memory("lmsg: ", fd, (unsigned long)lmsg);
report_memory("&lmsg: ", fd, (unsigned long)&lmsg);
report_memory("gmsg: ", fd, (unsigned long)gmsg);
report_memory("&gmsg: ", fd, (unsigned long)&gmsg);

report_memory("&buf: ", fd, (unsigned long)&buf);

for (i = 0; i<buffer_size; i++) {
buf[i] = (char *) malloc(4096);
snprintf(format, 16, "buf[%02d]: ", i);
report_memory(format, fd, (unsigned long)buf[i]);
}

report_memory("main: ", fd, (unsigned long)&main);
report_memory("puts: ", fd, (unsigned long)&puts);

return 0;
}
</syntaxhighlight>

===3000physicalview.c===

<syntaxhighlight lang="c" line>
/* 3000physicalview.c A kernel module to expose virtual->physical memory mappings in userspace as an ioctl
* Copyright (C) 2020 William Findlay
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>. */

/* Warning: This module is extremely insecure.
* It is designed purely for teaching purposes.
* Using it is stupid unless you are in COMP3000. */

#include "3000physicalview.h"

static struct device *device = NULL;
static struct class *class = NULL;
static int major_number;

/* Helper functions below this line ---------------- */

/* Walk the page table of current task for virtual address addr */
static unsigned long get_physical(unsigned long addr)
{
pgd_t *pgd;
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
pte_t *pte;

unsigned long pfn = 0;
unsigned long phys = 0;

/* Find pgd */
pgd = pgd_offset(current->mm, addr);
if (!pgd || pgd_none(*pgd) || pgd_bad(*pgd))
{
printk(KERN_ERR "Invalid pgd for address 0x%016lx\n", addr);
return phys;
}

/* Find p4d */
p4d = p4d_offset(pgd, addr);
if (!p4d || p4d_none(*p4d) || p4d_bad(*p4d))
{
printk(KERN_ERR "Invalid p4d for address 0x%016lx\n", addr);
return phys;
}

/* Find pud */
pud = pud_offset(p4d, addr);
if (!pud || pud_none(*pud) || pud_bad(*pud))
{
printk(KERN_ERR "Invalid pud for address 0x%016lx\n", addr);
return phys;
}

/* Find pmd */
pmd = pmd_offset(pud, addr);
if (!pmd || pmd_none(*pmd) || pmd_bad(*pmd))
{
printk(KERN_ERR "Invalid pmd for address 0x%016lx\n", addr);
return phys;
}

/* Find pte */
pte = pte_offset_map(pmd, addr);
if (!pte || pte_none(*pte))
{
printk(KERN_ERR "Invalid pte for address 0x%016lx\n", addr);
return phys;
}

/* Get physical address of page table entry */
pfn = pte_pfn(*pte);
phys = (pfn << PAGE_SHIFT) + (addr % PAGE_SIZE);

return phys;
}

/* Define file operations below this line ------------------- */

/* Callback to device open */
static int physicalview_open(struct inode * inode, struct file * file)
{
printk(KERN_INFO "3000physicalview open\n");
return 0;
}

/* Callback to device close */
static int physicalview_release(struct inode * inode, struct file * file)
{
printk(KERN_INFO "3000physicalview closed\n");
return 0;
}

/* Callback to device ioctl */
static long physicalview_ioctl(struct file *file, unsigned int cmd, unsigned long addr)
{
struct physicalview_memory *mem;
switch (cmd)
{
case PHYSICALVIEW_WALK:
/* Allocate kernel memory for our struct */
mem = kmalloc(sizeof(struct physicalview_memory), GFP_KERNEL);
if (!mem)
{
printk(KERN_ERR "Unable to allocate space for struct\n");
return -EFAULT;
}

/* Get virt from userspace */
if (copy_from_user(mem, (struct physicalview_memory *)addr,
sizeof(struct physicalview_memory)))
{
printk(KERN_ERR "Unable to copy struct from user\n");
kfree(mem);
return -EFAULT;
}

/* Call helper to get physical mapping for virtual address */
mem->phys = get_physical(mem->virt);

/* Give phys back to userspace */
if (copy_to_user((struct physicalview_memory *)addr, mem,
sizeof(struct physicalview_memory)))
{
printk(KERN_ERR "Unable to copy struct to user\n");
kfree(mem);
return -EFAULT;
}

/* Cleanup, cleanup, everybody do their share */
kfree(mem);
break;
default:
return -EINVAL;
}

return 0;
}

/* Register file operations */
static struct file_operations fops = {
.open = physicalview_open,
.release = physicalview_release,
.unlocked_ioctl = physicalview_ioctl,
};

/* World readable and writable because... security? */
static char *physicalview_devnode(struct device *device, umode_t *mode)
{
if (mode)
*mode = 0666;
return NULL;
}

/* Entry and exit points below this line ------------------------------------ */

/* Module initialization */
int init_module(void)
{
printk(KERN_INFO "3000physicalview initializing\n");

/* Register character device */
major_number = register_chrdev(0, DEVICE_NAME, &fops);
if (major_number < 0)
{
goto failed_chrdevreg;
}

/* Create device class */
class = class_create(THIS_MODULE, CLASS_NAME);
if (IS_ERR(class))
{
goto failed_classreg;
}

/* Set devnode to set our "super secure" permissions from above */
class->devnode = physicalview_devnode;

/* Create device */
device = device_create(class, NULL, MKDEV(major_number, 0), NULL, DEVICE_NAME);
if (IS_ERR(device))
{
goto failed_devreg;
}

printk(KERN_INFO "3000physicalview is loaded and regstered with major number %d!\n", major_number);

return 0;

/* NOTREACHED... */

/* Errors here */
failed_devreg:
printk(KERN_ERR "Failed to register 3000physicalview device!\n");
class_unregister(class);
class_destroy(class);
failed_classreg:
printk(KERN_ERR "Failed to register 3000physicalview class!\n");
unregister_chrdev(major_number, DEVICE_NAME);
failed_chrdevreg:
printk(KERN_ERR "Failed to register 3000physicalview as a character device!\n");
return -1;
}

/* Module destructor callback */
void cleanup_module(void)
{
/* Cleanup, cleanup, everybody do their share */
device_destroy(class, MKDEV(major_number, 0));
class_unregister(class);
class_destroy(class);
unregister_chrdev(major_number, DEVICE_NAME);

printk(KERN_INFO "3000physicalview cleanup complete\n");
}
</syntaxhighlight>

===3000physicalview.h===

<syntaxhighlight lang="c" line>
/* 3000physicalview.c A kernel module to expose virtual->physical memory mappings in userspace as an ioctl
* Copyright (C) 2020 William Findlay
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>. */

/* Warning: This module is extremely insecure.
* It is designed purely for teaching purposes.
* Using it is stupid unless you are in COMP3000. */

#ifndef PHYSICALVIEW_H
#define PHYSICALVIEW_H

#ifndef USERSPACE /* Kernelspace only */
#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/device.h>
#include <linux/string.h>
#include <linux/sched.h>
#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/uaccess.h>
#include <asm/uaccess.h>
#include <asm/pgtable.h>
#include <asm/pgtable_types.h>

#define DEVICE_NAME "3000physicalview"
#define CLASS_NAME "COMP3000"

MODULE_DESCRIPTION("Walk page table for userspace virtual address.");
MODULE_AUTHOR("William Findlay");
MODULE_LICENSE("GPL");
MODULE_VERSION("0.0.1");

#else /* Userspace only */
#include <sys/ioctl.h>
#endif

/* Both userspace and kernelspace */

struct physicalview_memory
{
unsigned long virt;
unsigned long phys;
};

/* Ioctl stuff */
#define PHYSICALVIEW_BASE 'k'
#define PHYSICALVIEW_WALK _IOWR(PHYSICALVIEW_BASE, 1, struct physicalview_memory)

#endif /* PHYSICALVIEW_H */
</source>

===Makefile===

<source lang="makefile" line>
obj-m += 3000physicalview.o

.PHONY: all, clean, insert, remove

all: 3000memview2 3000physicalview.ko

3000physicalview.ko: 3000physicalview.c 3000physicalview.h
make ARCH=x86_64 -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

3000memview2: 3000memview2.c
gcc -O2 -o 3000memview2 3000memview2.c

insert:
sudo insmod 3000physicalview.ko

remove:
sudo rmmod 3000physicalview

clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
rm 3000memview2
</syntaxhighlight>

Operating Systems 2022F: Tutorial 9 - Revision history

Soma: /* Watching Kernel Memory Allocations */