https://homeostasis.scs.carleton.ca/wiki/index.php?action=history&feed=atom&title=DistOS_2021F_2021-11-18DistOS 2021F 2021-11-18 - Revision history2026-05-12T23:28:38ZRevision history for this page on the wikiMediaWiki 1.42.1https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2021F_2021-11-18&diff=23507&oldid=prevSoma: Created page with "==Notes== <pre> Lecture 18 ---------- - proposals, I'm grading, that is my tomorrow - but tonight, I'll get you readings, hopefully for rest of term - experience 2 is almo..."2021-11-19T02:26:48Z<p>Created page with "==Notes== <pre> Lecture 18 ---------- - proposals, I'm grading, that is my tomorrow - but tonight, I'll get you readings, hopefully for rest of term - experience 2 is almo..."</p>
<p><b>New page</b></p><div>==Notes==<br />
<br />
<pre><br />
Lecture 18<br />
----------<br />
- proposals, I'm grading, that is my tomorrow<br />
- but tonight, I'll get you readings, hopefully for rest of term<br />
- experience 2 is almost done, should be released over the weekend<br />
- you'll have more time than we originally planned<br />
- probably till the final exam<br />
- he'll then get to grading experience 1<br />
<br />
Delos<br />
- why did I pick it? in part, "power of abstraction"<br />
- consensus protocols keep being used<br />
- many ones<br />
- each is very technical<br />
<br />
abstracting consensus to a shared log file<br />
- consensus over the order of events<br />
<br />
log files are central to maintaining state<br />
- in log-based filesystems, the log is everything<br />
- this is because the log determines how state is changed<br />
- if you can agree on the order of changes,<br />
(and what those changes are)<br />
you can agree on the final state<br />
<br />
Chubby said the central consensus mechanism should be a file<br />
- Delos says it should be a log<br />
- and you can use a log to make a consensus filesystem if you want!<br />
<br />
<br />
The abstraction says 'you have a log'<br />
- below: how do I make a log?<br />
- above: how do I use the log?<br />
<br />
<br />
DHTs & Tapestry<br />
<br />
Overlay networks<br />
- network on a network<br />
<br />
We have the Internet, why do we need anything else on top?<br />
<br />
an overlay network is adding a level of indirection to the networking problem<br />
- and that's how you solve problems in CS, right?<br />
<br />
You use an overlay network when you don't have an easy way of knowing the destination beforehand<br />
- where you communicate is based on what is being communicated<br />
<br />
On the Internet, I can find someplace by its IP address or DNS name<br />
- and if I don't know either, I use a search engine<br />
<br />
But a search engine is centralized<br />
- and we want a decentralized way<br />
<br />
So an overlay network is a way of routing information where<br />
- you don't know the location of the destination a-priori<br />
- and you don't have a centralized authority to tell you<br />
<br />
So we define a new topology, and the purpose of that topology is to facilitate the kind of information exchange we want to do<br />
- "neighbors" have a relationship that helps communication<br />
<br />
Social networks are a kind of overlay network<br />
- social graph overlayed on the Internet<br />
- but, mostly ends up just being a centralized service nowadays<br />
- but you could make a decentralized one<br />
<br />
On the Internet, how does data get from A to B?<br />
- A sends info to local router R_1<br />
- R_1 sends it to R_2<br />
- ...<br />
- R_n-1 sends it to R_n<br />
- R_n sends it to B<br />
<br />
So data goes from source to routers to destinations<br />
<br />
Routers know about each other, how to route data, using<br />
routing tables<br />
- spanning trees<br />
<br />
BGP is a protocol for updating routing tables<br />
<br />
So with an overlay network, we do the same thing<br />
- note that Tapestry even takes into account local topology<br />
<br />
We don't use this sort of stuff as much anymore<br />
- overlay networks are essential to any P2P system<br />
- but we don't use P2P so much nowadays<br />
<br />
Let's recall Skype, great example of an overlay & P2P network back in the day<br />
<br />
Before, Skype had very few central servers, yet it wanted to route calls for the planet<br />
- idea: make it a P2P network!<br />
- clients would become hosts, everyone would share in maintaining directory, routing calls<br />
- actual call would take place directly between two parties<br />
unless there were networking issues (firewalls, NAT)<br />
- but connecting the two parties, finding them via<br />
their usernames took serious resources<br />
<br />
Turns out university network admins HATED Skype<br />
- it would suck up their bandwidth<br />
- it was also stealthy, so they couldn't easily block it<br />
<br />
When Microsoft bought Skype, they dismantled the P2P network<br />
and replaced it with centralized servers<br />
- and made it much worse for end users<br />
<br />
What P2P apps do we use nowadays?<br />
- Bittorrent?<br />
- some games?<br />
- Windows update?<br />
<br />
but our home network connections are *so* much better than they were nowadays, and computers are so much more powerful<br />
- would be so easy to do P2P today<br />
<br />
mobile devices aren't the best for P2P<br />
- intermittent connectivity<br />
<br />
But the real problem is one of economics and sociology, not technology<br />
- P2P networks are heavily used for illegal activity<br />
- unclear how to monetize any P2P application, so who develops it?<br />
- nobody knows how to really stop the abuse of P2P systems<br />
- too easy for bad actors to mess everything up<br />
<br />
You could say that cryptocurrencies are the new P2P systems<br />
- but miners aren't something most people want to run<br />
</pre></div>Soma