Operating Systems 2020W: Tutorial 9

In this tutorial, we will be playing with eBPF in more detail than you have seen previously. Parts of this tutorial may be slightly confusing. That is okay. You are learning a completely new programming paradigm in one tutorial and you are not expected to master it. The primary goal here is to understand the difference between kernel modules and BPF programs, and be able to explain at a high level how they work.

Throughout this tutorial, you may wish to refer to the bcc documentation.

To get started, we will first examine the source code for 3000shellwatch.py and bpf_program.c, the userspace and kernelspace components of our BPF program respectively.

Getting Started

1. Open 3000shellwatch.py and bpf_program.c. Try to get an idea of the following (if you are stuck, check the documentation linked above):
   • What is a tracepoint? What is a kprobe? What is a uprobe? What are their similarities and differences? bpf_program.c contains examples of all three.
   • How do we pass events from kernelspace to userspace?
   • Do you notice any big differences between BPF programs and kernel modules? What things have you seen in kernel modules that are missing in the BPF program?
   • BPF programs are supposed to be completely production-safe due to the BPF verifier. Do you think it is possible to cause a kernel panic from a BPF program? Make a guess now, you will have a chance to test your guess later.
   • Run a familiar trace command of your choice from one of the previous tutorials, but append the -v flag to the end of the command. trace will now output the source code for the BPF program it generated. How does this output compare with the hand written BPF program in bpf_program.c? Do you think it would be possible to write one long trace command to do the same thing as 3000shellwatch.py?

Playing with 3000shellwatch

1. Open two terminals. In one terminal, run ./3000shell, and in another terminal run sudo ./3000shellwatch.py -p `pidof 3000shell`. Run a several commands in 3000shell and observe the output of 3000shellwatch.py. What system calls is 3000shell generating according to 3000shellwatch.py? Compare this output with that of strace.
2. Let's try to make 3000shellwatch.py crash the kernel. Modify the tracepoint on lines 69-95 to do something dangerous like dereferencing a NULL pointer. Run your BPF program. What is all that output? Did you crash your kernel or did something else happen?
3. Can you use kernelspace helper functions in BPF programs? Try including a header file and calling a kernel function like copy_to_user. What happens when you try to run your program?
4. Could you write a kernel module that does everything our BPF program does? How hard would this be?
5. Optional: See if you can implement your own tracepoint in 3000shellwatch. To do this, you need to do the following:
   • Examine the list of available tracepoints with sudo tplist. You can search for specific strings by providing it an optional argument. When you find one that looks promising, you can view it in detail by passing the -v flag. If you're looking for an easy suggestion, try syscalls:sys_enter_write.
   • Add your own struct definition that will contain data from the event.
   • Add your own perf event buffer that will pass event data to userspace.
   • Using the same syntax as the existing tracepoint (lines 69-95), add in the definition of your tracepoint.
   • Using the same syntax from the perf buffers in the userspace python script (e.g. lines 25-29), attach your perf buffer so that it will produce output.