Adaptive Security 2020F Lecture 1

Course description

Advanced computer security course on technologies and approaches that are robust against adaptive adversaries. Topics include AI, machine learning, artificial life, feedback control, randomization, and diversity as applied to computer security. Course project will involve students modelling, testing, and/or implementing their own adaptive defenses.

Learning Objectives

In this course you will learn how to critique and evaluate adaptive security mechanisms, including ones that use diversity/randomness, machine learning, and other adaptive approaches. You will also get experience in designing adaptive security mechanisms, and you will learn the advantages and disadvantages of adaptive approaches to security.

Course logistics

• We will meet virtually 8:35-9:55 Tuesday and Thursday during the fall term. For now we will meet via zoom using the link on cuLearn, however we may change that.

• The meeting time is a bit early, but we need to stick with it for at least the first few weeks until enrollment stabilizes. Once it does we can potentially change the time by mutual consensus.

• We will also communicate asynchronously via Microsoft Teams, Discord, or Slack. Any should work, let's discuss!

Grading Scheme

• 10% Participation
• 20% Research journal
• 70% Project

Participation

• You are expected to attend most class meetings and participate in asynchronous class discussions.
• Students making a genuine effort to participate will earn most of these marks (with a bit reserved for truly exceptional participation).

Research Journal

For this class you'll maintain an electronic research journal which will detail your ongoing thoughts related to this class. You should record your:

• responses to assigned readings
• ideas that arise from class
• brainstorming for your class project
• and any other relevant thoughts.

The instructor will review your journal before each class.

You may use any easily shareable application to maintain your journal.

• Journals will be graded as a participation grade, with marks assigned out of 4 for each class.*

Project

• In this course you'll be doing a term project that will produce a publishable research paper or a research proposal. This project has multiple milestones.
• You will first submit an areas of interest document that outlines the kinds of projects and the types of software systems you wish to work with.
• Then, you'll submit an "elevator pitch" which will be a few paragraph summary of your idea, concisely communicating the basic idea for your project.
• You'll then submit a literature review related to your idea and a report on preliminary findings/tests
• At the end of the term you'll present your work to the class and,
• During the exam period you'll submit your completed paper or proposal.

Project Grading & Milestones

• 2% areas of interest (September 18, 2020)
• 3% elevator pitch (September 29, 2020)
• 20% literature review (October 23, 2020)
• 10% tests/preliminary work (November 17, 2020)
• 5% presentation (December 8 & 10, 2020)
• 30% research proposal/paper (Exam period)

Collaboration & Plagiarism

• You're allowed to to discuss class material and your projects with others in the class.
• What you submit, however, should be your own work. *Any* outside contributions should be clearly acknowledged.
• Intellectual dishonesty fundamentally undermines research. Behave accordingly.

Accommodations & University Policies

• While I expect you to take this class seriously, it isn't the most important thing in your life or mine.
• Life intervenes. I will do my best to accommodate when that happens.
• I can give extra time but I can't lower requirements.

Course hypothesis

• Most current defenses
  • don't take into account inevitable attacker adaptation and
  • have limited in-built means to respond to attacker innovation.
• As a result, *people* must constantly be responding to attacker changes.
• But this game is not symmetric.
  • Attackers force defenders to put in proportionately more effort.
• So no matter how strong individual mechanisms and systems are, defenders are losing the security meta game.
• If we are to change this, our defensive strategies and mechanisms must be robust to attacker innovation. If there is an asymmetry between necessary attacker and defender effort, it should favor the defender, not the attacker.

Adaptive Security

Adaptive security is my term for defensive security strategies and mechanisms that are robust to attacker innovation.

Others have used this term. Look it up. It makes for depressing reading.

What is "Security"?

• Standard view:, Confidentiality, Integrity, Availability
• But *why* do these matter?
• Security is about being ready for threats.
• What is a threat? Things that can destroy you.
• Security thus is ultimately about survival.

What is "Adaptive"?

• Something is adaptive if it helps you adapt to changing circumstances.
• Standard defenses are mostly not adaptive
  • Cryptography
  • Access control
  • Code injection defenses
• What strategies are fundamentally adaptive?

Adaptation versus perfection

• Our goal here is not to make systems that are perfectly secure.
• Instead, we want to make systems that can keep up in the eternal arms race with attackers without requiring every human on the planet to work to defend them.
• Most existing approaches are based on an implicit assumption that we can "get it right" - but we never do.
  • Consider cryptographic and software vulnerabilities - we keep finding them, and once found they undermine all security guarantees
• The current arms race is unsustainable; adaptive security strategies and mechanisms could change this.

Biology and Adaptive Security

My ideas on adaptive security are informed by observation of the defensive strategies of living systems.

Because the implementation substrate of modern computers is so different from that of cells, biological defenses are not that informative at a mechanistic level (although there are exceptions). Living systems do have so much to tell us at an architectural and strategic level, however.

Core Ideas

• interconnected feedback loops
• variation

Feedback loops

• Interconnected feedback loops are how everything works in biology. Sensors and effectors are intertwined to form complex regulatory networks where changes anywhere can effect the entire network. These changes can be dampened so overall behavior hardly changes; but they can also be amplified so a small change leads to a phase transition in the network.

• We don't engineer systems with interconnected feedback loops if we can avoid it, because the resulting nonlinear dynamics make systems fundamentally hard to debug. As we will see, avoiding them leads to systematic weaknesses.

Variation

• Evolution is just variation plus selection. Lots of attention is paid to selection, e.g. "survival of the fittest". But the really important part of this is the variation.

• Living systems survive over time through variation. Variation allows the search space of possibilities to be explored, e.g., so better solutions to environmental challenges can be found.

• But from a security perspective, variation it itself is a defensive strategy, because it means that attackers have to constantly follow. A stationary target is fundamentally easier to hit, and the longer it is stationary the more opportunity an attacker has to optimize. A moving target reduces attacker optimization opportunities.

Modern systems have low levels of variation and make little use of feedback loops. As a result, attackers have ample time to optimize their attacks, and compromise attempts, successful or failed, are barely noticed or responded to because the only feedback loops that could do so involve people.

In adaptive security, we're trying to change this game. And I want you to be part of it.

What we're doing here

Here's the big secret: feedback loops and variation aren't hard to add to current systems. As a community, however, we actively avoid both due to other factors.

In this class we will explore how feedback loops and variation can be added and the factors that make such additions challenging. My goal it for you to rethink what is possible in the realm of security mechanisms.

Machine Learning

• Note that I haven't made a mention of machine learning in the context of adaptive security, and there is a reason for it.
• Most of the time, current approaches to machine learning are not applicable to making really adaptive mechanisms.
• This might seem to be a paradox, as the most popular machine learning algorithms, ones for "deep learning", are supposed to be biologically inspired, just as adaptive security is.
• Note that learning can be adaptive...but it can also be maladaptive.
  • We can learn the wrong lessons
• The problem is deep learning and neural networks are inspired by the structure of a biological system (namely, the brain), but misses its architectural and strategic factors.
• In other words, deep learning copies the idea of interconnected neurons but misses the feedback loops and variation that are key to making our brains do what they do.
• As a result, systems built on deep learning technology are often more maladaptive than adaptive.

Adaptive Security in context

• These ideas draw upon cybernetics, artificial life, and complex adaptive systems.
• But, adaptive security is something fundamentally new because it brings somethings these other fields never really captured: the fight for survival.
• In computer security, we're dealing with powerful adaptive adversaries who don't follow any fixed set of constraints. There's no set rules of engagement---anything goes.
• Living systems are complex, convoluted things that only make sense when we understand they were shaped by the forces of evolution.
• Modern computer systems are also constantly changing in response to evolutionary pressures.
• As long as we don't understand the meta-game we are playing, though, those pressures can just as well lead to extinction rather than survival.

Plan for the semester

• Artificial Life
• Non-adaptive deployed systems
• Adaptive deployed systems
• Adaptive research systems
• Your ideas for adaptive systems!

Questions?