DistOS 2023W 2023-03-20

Notes

March 20
--------

Project Proposals
 - if you want to re-submit, you can, and I will replace the grade with your new one
   - Need to finish it by March 27th

 - some proposals were just way too broad
   - need to narrow it down
 - some were too focused on one paper
   - needs to cover multiple papers
 - some tried to argue for a technical point
   - you aren't doing original research
   - focus on what the papers say, have a thesis about patterns
     in their arguments

Remember the thesis comes after you've read and understood the papers
 - represents some of the insight you got from the papers

Borg & Omega

A container
 - packaged Unix-like userland
   - generally to be run on a Linux kernel
   - only dependency is on the Linux kernel, all
     userspace dependencies are included
 - popular in the cloud space because allows for composition of workloads on a host
   - can run without seeing each other with OS virtualization
     (private filesystem, user ID and process IDs, resource quotas, etc)
 - note that containers don't *really* contain from a security perspective
   - must use advanced OS security mechanisms (SELinux, system call filtering)
     to keep them separate, even then generally not enough
   - if running untrusted workloads, generally use hardware virtual machines to separate
   - area of my research how to do better! (look up bpfcontain, bpfbox)
   - remember, virtualization is not confinement!
     (virtualization is like putting on VR goggles, doesn't change how
      you can affect the world just what you see)


Remember with Google (Borg, Omega), it is running containers that are trusted (i.e., they made them all), so they don't have to worry about containers going bad except if they are compromised, and they have other mechanisms to deal with that

AFTER GROUP DISCUSSION

What is Borg?
 - container orchestration
 - foundation for Google's distributed OS
 - unit of resource management is the container, not the process
   - notice the fate of UNIX, it isn't the operating system,
     it became the new process (rather than make a distributed UNIX)

So they had Borg, why did they make Kubernetes?

How did they make Borg?
 - Borg is old and dates from the early days of Google
    - didn't rely on hardware virtualization since their systems didn't support it!
 - so it is full of design choices that made sense at the time, but maybe don't now
   - for example, only one IP address per host
 - Borg was made specifically for Google, co-evolved with it
   - doesn't really make sense outside of their environment


Kubernetes is the Google engineers getting a second chance to build Borg "right"
 - what they want to use
 - but really, it is a bit *too* general, can handle
   Google-scale workloads but very few need that much scalability
   and the cost is complexity

What is Omega?
 - research project to do better scheduling than Borg
 - I'm sure many of the ideas have been put into Borg by now

Scheduling is a classic OS problem
 - lots of theoretical result
 - but in practice schedulers are quite messy
   - need domain-specific knowledge to decide what is important and
     to understand its needs
   - will tend to have heterogeneous workloads