https://homeostasis.scs.carleton.ca/wiki/api.php?action=feedcontributions&user=Soma&feedformat=atomSoma-notes - User contributions [en]2024-03-29T01:09:32ZUser contributionsMediaWiki 1.37.1https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-04-05&diff=24426DistOS 2023W 2023-04-052023-04-05T21:54:03Z<p>Soma: /* Notes */</p>
<hr />
<div>==Class Discussion==<br />
<br />
For class discussion today:<br />
* What did we learn this semester?<br />
* Specifically, what are the big ideas, the patterns we covered?<br />
<br />
==Notes==<br />
<br />
<pre><br />
Exam Review<br />
-----------<br />
<br />
EXAM LOGISTICS<br />
<br />
Exam is on April 21, 2-5 PM (COMP 4000)<br />
<br />
(COMP 5102 is scheduled on April 27, 2-5 PM, but please take the exam on April 21 unless you have a good reason. I will need to manually open up the exam for you on April 27th.)<br />
<br />
Exam is online just like the midterm, same format, but three hours.<br />
<br />
Projects are due on April 27th by midnight. I have until May 1st to get grades in.<br />
<br />
Exam will require three essays from 3 or more questions (I haven't made it up yet). Final exam is cumulative.<br />
<br />
<br />
PRESENTATION LOGISTICS<br />
<br />
To facilitate presentations on Monday, please send me your slides by Sunday night as a PDF file, via email (my cunet address) or Teams. You may have at most 4 slides (including title), plan to talk for 3 minutes. I will record so I can grade after class.<br />
<br />
Be sure to present the thesis of your lit review and give evidence in support of the thesis by referring to specific papers/systems<br />
<br />
EVERYONE SHOULD COME TO CLASS ON MONDAY.<br />
- will probably have topics of relevance to the final exam<br />
<br />
DISCUSSION SUMMARY<br />
<br />
What was this class about?<br />
<br />
"distributed abstractions for application development"<br />
<br />
Why are Ceph and Spanner similar?<br />
- both take old APIs that were hard to scale and make them scalable<br />
(UNIX files & SQL)<br />
- but at the cost of complexity<br />
<br />
When studying for the exam, you have to go beyond labels<br />
- what are the real commonalities? What are the patterns?<br />
- really try to compare papers and see how they relate<br />
<br />
What abstractions work in a distributed context, and how can those abstractions be implemented?<br />
<br />
Ongoing considerations<br />
- how to manage state? Make it immutable wherever you can (append-mostly files), and turn mutable state into immutable state everywhere else (e.g., use logs)<br />
<br />
- minimise consistency requirements, but don't be afraid to use strong consistency where needed using consensus protocols such as Paxos or Raft<br />
<br />
- build for failure of hosts, this will happen, so no computer can be essential, all must be replacable<br />
<br />
- abstract the OS away from the hardware to facilitate distribution of load (e.g., containers)<br />
<br />
- (scheduling is hard and generally requires domain knowledge of workload<br />
(lots of hacks in practice))<br />
<br />
- keep things simple initially, but later don't be afraid to make the system complex to support easier/more conventional abstractions for developers (users of the system)<br />
<br />
- abstractions for distributed applications need to be specialized to be performant (because parallel is hard)<br />
<br />
- Much better performance is possible if you're running trusted workloads<br />
- compare BOINC, OceanStore with GFS, Dynamo, MapReduce, etc<br />
- BOINC vs MapReduce<br />
- OceanStore vs f4<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_23&diff=24425Mobile Apps 2023W Lecture 232023-04-05T21:53:37Z<p>Soma: Created page with "==Notes== <pre> Final lecture ------------- "Development of applications for mobile environments taking advantage of gesture-based input and using location and presence services. Topics include introduction to low-level network services and mobile platforms, description of architectural patterns, principles of mobile development and interaction styles for network service usage." We covered most of this. Location and presence services are pretty straightforward use of..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Final lecture<br />
-------------<br />
<br />
"Development of applications for mobile environments taking advantage of gesture-based input and using location and presence services. Topics include introduction to low-level network services and mobile platforms, description of architectural patterns, principles of mobile development and interaction styles for network service usage."<br />
<br />
We covered most of this. Location and presence services are pretty straightforward use of standard Android APIs or REST APIs.<br />
<br />
<br />
Focus for today - where to go from here<br />
<br />
Mobile development is programming using a powerful set of proprietary, ever changing, under documented APIs.<br />
- and most of this is pretty normal<br />
- only thing that sometimes changes is the "proprietary" part<br />
<br />
Modern programming is all about adding your little bit of code to a mass of already existing code<br />
<br />
<br />
So what are the most relevant courses to mobile development?<br />
- COMP 2406 (web dev) - web technology should be used where you can because that's cross platform, and many mobile APIs are based on existing web standards or frameworks<br />
- and in fact, use web technologies unless you have a specific need to not do so, and even then think again<br />
<br />
- COMP 3007 (programming paradigms) - functional, declarative, and even logic programming are increasingly relevant and used, even if they are disguised<br />
- because state is very hard to manage in a distributed context<br />
- if you're making a mobile app, it will generally be "distributed" in some way and so make state difficult to manage, even if it runs completely on the device (e.g., multicore)<br />
- mutating data structures is very hard to scale<br />
- better to avoid when you can<br />
<br />
- COMP 3000 (operating systems) - need to understand the underlying computation model if you want to understand performance, reliability, and security concerns<br />
<br />
- COMP 3005 (databases) - databases are a tool for concurrent state management. If you're managing large amounts of state in any non-trivial app, you'll probably want a database or two<br />
<br />
- COMP 3008 (HCI) - human factors matter in any modern app, if you don't understand the principles you'll make apps that are very frustrating to users or even insecure<br />
<br />
- COMP 3004 - how to architect and build larger programs, with other people!<br />
<br />
Because the hard part of doing mobile dev is understading how things work. And you need to have the right conceptual framework for that, which is what you get from these courses.<br />
<br />
So how valuable is learning iOS or Android development?<br />
- marketable skill certainly<br />
- but will likely go stale if not kept refreshed<br />
<br />
- can get you exposed to a lot of ideas about how to build programs in a concentrated form<br />
- concentrated "wisdom" (patterns) of experienced developers, ideas on how programs should be built<br />
- these patterns can help you understand the code of others, once you understand them<br />
<br />
But I know this has been frustrating for a lot of you. I'll just say...it is frustrating for everyone except people who do this all the time. And for them, they were frustrated until they learned all the tricks.<br />
<br />
And I will say that once you gain experience in one of these environments, it gets *much* easier to work in others<br />
- because the underlying ideas are often the same, certainly lots of sharing<br />
<br />
<br />
Remember that if you think things should be done differently, that doesn't mean you are wrong. Learn how others do it, and when you can try doing it your own way. You'll then learn whether your ideas are good or not!<br />
- everything you work with was built by someone who at some point didn't know what they were doing, and to this day probably think they still don't mostly know what they are doing (they know a lot but know there's lots more they don't know)<br />
<br />
Another thing you should try learning: security<br />
- not just cryptography, but that is important<br />
- more generally, about how attackers break systems and how you can strengthen your apps<br />
- because almost any app today is subject to security issues because they are on the Internet<br />
<br />
In mobile development, code signing is very important<br />
- you can't install an app on a device if it isn't properly signed<br />
- so you generally have to generate a certificate before installing an app<br />
- Apple's dev tools make it clear this is what is happening<br />
- I know Android needs apps to be signed but I'm not sure how<br />
Android Studio does it<br />
- I think it is just using a dummy key and telling Android to trust it<br />
- iOS is stricter, esp as installed apps expire<br />
<br />
<br />
My opinion is that longer term, development, even for mobile devices, isn't going to be based on proprietary stacks. Instead, we're going to use web-based technologies wherever we can.<br />
- but it won't just be JavaScript, it will be be built on WebAssembly<br />
- because the code will be sandboxed, portable, and GUI-enabled<br />
- relatively few apps will need anything more, and they will often use web technology for most of their functionality<br />
<br />
Notice the cloud is built almost entirely on open source, non-proprietary technologies<br />
- best developer experience<br />
- same will happen with mobile, eventually, as platforms mature<br />
<br />
Android as a set of APIs is only useful to learn for job prospects. But the concepts underlying them are mostly pretty universal so are worth learning, those ideas will show up elsewhere.<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24424Mobile Applications (Winter 2023)2023-04-05T21:53:15Z<p>Soma: /* April 5, 2023 */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|Webview history 1]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|Webview history 2]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|Class wrap-up]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 26, 2023===<br />
<br />
Final Projects due<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-04-05&diff=24423DistOS 2023W 2023-04-052023-04-05T15:41:37Z<p>Soma: Created page with "==Class Discussion== For class discussion today: * What did we learn this semester? * Specifically, what are the big ideas, the patterns we covered? ==Notes== <pre> </pre>"</p>
<hr />
<div>==Class Discussion==<br />
<br />
For class discussion today:<br />
* What did we learn this semester?<br />
* Specifically, what are the big ideas, the patterns we covered?<br />
<br />
==Notes==<br />
<br />
<pre><br />
<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-04-03&diff=24422DistOS 2023W 2023-04-032023-04-03T17:52:40Z<p>Soma: /* Notes */</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Spanner & Tensorflow<br />
--------------------<br />
<br />
Last two papers!<br />
<br />
April 5th - class wrap-up discussion, exam review<br />
April 10 - project presentations<br />
<br />
Spanner - big, distributed SQL database (mostly)<br />
- at Google<br />
- compare with Bigtable, Dynamo (NoSQL systems)<br />
- what is the difference in functionality?<br />
- why does it matter?<br />
- HOW?! what is the "neat trick"?<br />
- has to do with time, but why?<br />
- to what degree is Spanner a full relational database, like Postgres?<br />
<br />
Tensorflow<br />
- how does it compare with mapreduce?<br />
- what type of operations does it handle?<br />
- why are these operations important?<br />
- how are they done in a scalable fashion?<br />
- why isn't this more general?<br />
<br />
<br />
AFTER DISCUSSION<br />
<br />
Only presentations on April 10th, no class on April 12th<br />
<br />
If you wish to volunteer to present on April 5th, please PM me<br />
- time left for speakers on the 10th will depend on how many volunteer on the 5th<br />
- may have "lightning talks" of 3-4 min<br />
<br />
Thoughts on Spanner?<br />
<br />
So why spanner?<br />
- wanted to support more complex queries (i.e., ones involving multiple tables)<br />
- programmers are used to using SQL<br />
- and transactions<br />
- great for updates without corrupting data (i.e., partial, incomplete updates)<br />
<br />
SQL is the obvious choice for making a database<br />
- compromised to support scalability<br />
- how does Spanner do SQL & scalability?<br />
<br />
True time is just a way to get really accurate time<br />
- generally from GPS<br />
<br />
GPS turns time information into location<br />
- only works with super accurate clocks<br />
- GPS satellites are just atomic clocks that broadcast their time<br />
<br />
So anyway any computer with a GPS receiver has access to very accurate clocks<br />
<br />
So put them in servers, they have accurate time, and now we can get an absolute ordering of events<br />
<br />
Normally you can't use local clocks to determine relative order of events because of clock skew.<br />
- but with really accurate clocks, you can!<br />
<br />
Traditionally databases were the hardest part of a web app to scale<br />
- but now we really can scale them as far as we want<br />
<br />
Tensorflow<br />
<br />
parallel computing is hard<br />
- especially when communication is slow/expensive<br />
<br />
Distributed OS is largely trying to hide the complexity of parallel computing<br />
from apps<br />
- provide the right abstractions<br />
<br />
POSIX files, processes, they aren't right<br />
<br />
But append-only files, containers, and specialized computation abstractions are<br />
- mapreduce, BOINC were our first examples, required embarassingly parallel problems<br />
- tensorflow is an effort to parallelize machine learning apps in a general way<br />
- abstraction: dataflow<br />
<br />
So why tensorflow?<br />
- dataflow with tensors<br />
- tensors are just multidimensional arrays<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24421Mobile Applications (Winter 2023)2023-04-03T15:53:49Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|Webview history 1]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|Webview history 2]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 26, 2023===<br />
<br />
Final Projects due<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Distributed_OS:_Winter_2023&diff=24420Distributed OS: Winter 20232023-04-03T15:53:06Z<p>Soma: </p>
<hr />
<div>==Course Outline==<br />
<br />
[[Distributed OS: Winter 2023 Course Outline|Here]] is the course outline.<br />
<br />
==Project Help==<br />
<br />
To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.<br />
<br />
To begin selecting a paper, I suggest that you:<br />
* search on Google Scholar using keywords relating to your interests, and/or<br />
* browse the proceedings of major conferences that publish work related to distributed operating systems.<br />
<br />
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].<br />
<br />
To help you write a literature review or the background of a research paper, read the following:<br />
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]<br />
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]<br />
<br />
==Class Schedule & Readings==<br />
<br />
===[[DistOS 2023W 2023-01-09|January 9, 2023]]===<br />
<br />
Introduction<br />
<br />
===[[DistOS 2023W 2023-01-11|January 11, 2023]]===<br />
<br />
Designing a distributed operating system<br />
<br />
===[[DistOS 2023W 2023-01-16|January 16, 2023]]===<br />
<br />
The Early Internet & Multics:<br />
* [https://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]<br />
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video<br />
Optional: Browse around [https://www.multicians.org/ the Multicians website].<br />
<br />
===[[DistOS 2023W 2023-01-18|January 18, 2023]]===<br />
<br />
UNIX<br />
* [https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/ Richard Jensen, "Unix at 50: How the OS that powered smartphones started from failure" (2019, Ars Technica)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]<br />
* [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video<br />
Note that the video covers the main points of the UNIX paper.<br />
<br />
===[[DistOS 2023W 2023-01-23|January 23, 2023]]===<br />
<br />
The Mother of All Demos:<br />
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].<br />
* [https://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]<br />
<br />
The Alto:<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])<br />
<br />
===[[DistOS 2023W 2023-01-25|January 25, 2023]]===<br />
<br />
Distributed Shared Memory<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/kai-li-shared.pdf Kai Li, "Shared virtual memory on loosely coupled multiprocessors" (1986)] '''(OPTIONAL)'''<br />
<br />
===[[DistOS 2023W 2023-01-30|January 30, 2023]]===<br />
<br />
Remote Procedure Calls<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/nelson1981-rpc.pdf Bruce J. Nelson, "Remote Procedure Call" (1981)] (Read the summary starting on p. 224, look at the Intro and Background if you have time.)<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/rfc1050.txt Sun Microsystems, "RPC: Remote Procedure Call Protocol Specification" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-01|February 1, 2023]]===<br />
<br />
LOCUS & NFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]<br />
<br />
===[[DistOS 2023W 2023-02-06|February 6, 2023]]===<br />
<br />
Sprite, AFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-08|February 8, 2023]]===<br />
<br />
Plan 9 & Inferno<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2018f/doward97-inferno.pdf Doward et al., The Inferno Operating System (1997)]<br />
* [http://www.inferno-os.info/inferno/ Inferno OS website] (browse)<br />
* [http://doc.cat-v.org/inferno/4th_edition/styx Rob Pike and Dennis M. Ritchie, "The Styx Architecture for Distributed Systems"]<br />
<br />
===[[DistOS 2023W 2023-02-13|February 13, 2023]]===<br />
<br />
Midterm review<br />
<br />
===February 15, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/~soma/distos/2023w/comp4000-2023w-midterm.pdf Midterm exam]<br />
<br />
===[[DistOS 2023W 2023-02-27|February 27, 2023]]===<br />
<br />
"Web Scale" discussion<br />
<br />
===[[DistOS 2023W 2023-03-01|March 1, 2023]]===<br />
<br />
OceanStore & BOINC<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]<br />
<br />
Project Proposal due March 3rd<br />
<br />
===[[DistOS 2023W 2023-03-06|March 6, 2023]]===<br />
<br />
NASD & Tapestry<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]<br />
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]<br />
<br />
Background (optional but helpful):<br />
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]<br />
<br />
===[[DistOS 2023W 2023-03-08|March 8, 2023]]===<br />
<br />
Chubby & GFS<br />
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]<br />
* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]<br />
<br />
===[[DistOS 2023W 2023-03-13|March 13, 2023]]===<br />
<br />
Ceph<br />
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]<br />
<br />
===[[DistOS 2023W 2023-03-15|March 15, 2023]]===<br />
<br />
BigTable & MapReduce<br />
* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]<br />
* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)<br />
<br />
===[[DistOS 2023W 2023-03-20|March 20, 2023]]===<br />
<br />
Omega & Borg<br />
* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]<br />
* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]<br />
<br />
===[[DistOS 2023W 2023-03-22|March 22, 2023]]===<br />
<br />
Cassandra & Dynamo<br />
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]<br />
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]<br />
<br />
===[[DistOS 2023W 2023-03-27|March 27, 2023]]===<br />
<br />
Zookeeper & Delos<br />
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]<br />
* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]<br />
<br />
===[[DistOS 2023W 2023-03-29|March 29, 2023]]===<br />
<br />
Haystack & F4<br />
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]<br />
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]<br />
<br />
===[[DistOS 2023W 2023-04-03|April 3, 2023]]===<br />
<br />
Spanner & Tensorflow<br />
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]<br />
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]<br />
<br />
===[[DistOS 2023W 2023-04-05|April 5, 2023]]===<br />
<br />
Class wrap up, final exam review<br />
<br />
===April 10, 2023===<br />
<br />
Project Presentations</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-04-03&diff=24419DistOS 2023W 2023-04-032023-04-03T15:47:59Z<p>Soma: Created page with "==Notes== <pre> Spanner & Tensorflow -------------------- Last two papers! April 5th - class wrap-up discussion, exam review April 10 & 12 - project presentations Spanner - big, distributed SQL database (mostly) - at Google - compare with Bigtable, Dynamo (NoSQL systems) - what is the difference in functionality? - why does it matter? - HOW?! what is the "neat trick"? - has to do with time, but why? - to what degree is Spanner a full relational database, l..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Spanner & Tensorflow<br />
--------------------<br />
<br />
Last two papers!<br />
<br />
April 5th - class wrap-up discussion, exam review<br />
April 10 & 12 - project presentations<br />
<br />
Spanner - big, distributed SQL database (mostly)<br />
- at Google<br />
- compare with Bigtable, Dynamo (NoSQL systems)<br />
- what is the difference in functionality?<br />
- why does it matter?<br />
- HOW?! what is the "neat trick"?<br />
- has to do with time, but why?<br />
- to what degree is Spanner a full relational database, like Postgres?<br />
<br />
Tensorflow<br />
- how does it compare with mapreduce?<br />
- what type of operations does it handle?<br />
- why are these operations important?<br />
- how are they done in a scalable fashion?<br />
- why isn't this more general?<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_22&diff=24418Mobile Apps 2023W Lecture 222023-03-31T22:05:56Z<p>Soma: Created page with "==Code== <syntaxhighlight lang="kotlin" line> package carleton.comp2601.webviewcompose import android.os.Bundle import android.util.Log import android.view.ViewGroup import android.webkit.WebResourceRequest import android.webkit.WebView import android.webkit.WebViewClient import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.compose.foundation.layout.Row import androidx.compose.foundation.text.ClickableText import androi..."</p>
<hr />
<div>==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.webviewcompose<br />
<br />
import android.os.Bundle<br />
import android.util.Log<br />
import android.view.ViewGroup<br />
import android.webkit.WebResourceRequest<br />
import android.webkit.WebView<br />
import android.webkit.WebViewClient<br />
import androidx.activity.ComponentActivity<br />
import androidx.activity.compose.setContent<br />
import androidx.compose.foundation.layout.Row<br />
import androidx.compose.foundation.text.ClickableText<br />
import androidx.compose.material.*<br />
import androidx.compose.runtime.*<br />
import androidx.compose.ui.graphics.Color<br />
import androidx.compose.ui.text.AnnotatedString<br />
import androidx.compose.ui.tooling.preview.Preview<br />
import androidx.compose.ui.viewinterop.AndroidView<br />
<br />
class MainActivity : ComponentActivity() {<br />
<br />
override fun onCreate(savedInstanceState: Bundle?) {<br />
super.onCreate(savedInstanceState)<br />
setContent {<br />
// Calling the composable function<br />
// to display element and its contents<br />
MainContent()<br />
}<br />
}<br />
}<br />
<br />
// Creating a composable<br />
// function to display Top Bar<br />
@Composable<br />
fun MainContent() {<br />
var curURL by remember { mutableStateOf("https://news.ycombinator.com") }<br />
var history = mutableListOf<String>("https://news.ycombinator.com")<br />
var curURLIndex = 0<br />
<br />
fun updateURL(newURL: String) {<br />
curURLIndex += 1<br />
history.add(curURLIndex, newURL)<br />
val lastIndex = history.lastIndex<br />
if (lastIndex > curURLIndex) {<br />
for (i in curURLIndex + 1..lastIndex) {<br />
history.removeAt(i)<br />
}<br />
}<br />
curURL = newURL<br />
Log.d("webviewCompose", "updating URL to $curURL")<br />
}<br />
<br />
fun prevURL() {<br />
curURLIndex -= 1<br />
if (curURLIndex < 0) {<br />
curURLIndex = 0<br />
}<br />
curURL = history[curURLIndex]<br />
Log.d("webviewCompose", "going to previous URL $curURL")<br />
}<br />
<br />
fun nextURL() {<br />
curURLIndex += 1<br />
if (curURLIndex > history.lastIndex) {<br />
curURLIndex = history.lastIndex<br />
}<br />
curURL = history[curURLIndex]<br />
Log.d("webviewCompose", "going to next URL $curURL")<br />
}<br />
<br />
Scaffold(<br />
// topBar = { TopAppBar(title = {Text("COMP 2601")})},<br />
topBar = {<br />
Row {<br />
ClickableText(<br />
AnnotatedString(text = "Back"),<br />
onClick = {<br />
prevURL()<br />
}<br />
)<br />
ClickableText(<br />
AnnotatedString(text = "Forward"),<br />
onClick = {<br />
nextURL()<br />
}<br />
)<br />
URLBar(curURL = curURL, updateURL = ::updateURL)<br />
}<br />
},<br />
content = { MyContent(curURL = curURL, updateURL = ::updateURL) }<br />
)<br />
}<br />
<br />
@Composable<br />
fun URLBar(curURL: String, updateURL: (String) -> Unit) {<br />
var newURL = ""<br />
<br />
TextField(<br />
value = curURL,<br />
//onFocusChange = { updateURL(newURL) },<br />
//onValueChange = { updatedURL: String -> newURL = updatedURL},<br />
onValueChange = updateURL,<br />
label = { Text("Enter URL here") }<br />
)<br />
}<br />
<br />
@Composable<br />
fun MyContent(curURL: String, updateURL: (String) -> Unit) {<br />
<br />
class ObservableWebViewClient : WebViewClient() {<br />
<br />
override fun shouldOverrideUrlLoading(view: WebView?, request: WebResourceRequest): Boolean {<br />
val url = request.url<br />
<br />
if (!request.isRedirect) {<br />
updateURL(url.toString())<br />
}<br />
return false<br />
}<br />
}<br />
<br />
fun logWebViewHistory(view: WebView) {<br />
val history = view.copyBackForwardList()<br />
<br />
val current = history.currentIndex<br />
<br />
Log.d("webviewCompose", "Current history item: $current")<br />
<br />
Log.d("webviewCompose", "Webview BackForwardList contents:")<br />
<br />
for (i in 0..history.size -1) {<br />
val item = history.getItemAtIndex(i)<br />
<br />
if (item != null) {<br />
val u = item.originalUrl<br />
Log.d("webviewCompose", " $i: $u")<br />
}<br />
}<br />
}<br />
<br />
// Adding a WebView inside AndroidView<br />
// with layout as full screen<br />
AndroidView(factory = {<br />
WebView(it).apply {<br />
// layoutParams = ViewGroup.LayoutParams(<br />
// ViewGroup.LayoutParams.MATCH_PARENT,<br />
// ViewGroup.LayoutParams.MATCH_PARENT<br />
// )<br />
webViewClient = ObservableWebViewClient()<br />
loadUrl(curURL)<br />
}<br />
}, update = {<br />
Log.d("webviewCompose", "loading URL $curURL")<br />
it.loadUrl(curURL)<br />
//logWebViewHistory(it)<br />
})<br />
}<br />
<br />
// For displaying preview in<br />
// the Android Studio IDE emulator<br />
@Preview(showBackground = true)<br />
@Composable<br />
fun DefaultPreview() {<br />
MainContent()<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24417Mobile Applications (Winter 2023)2023-03-31T22:05:08Z<p>Soma: /* March 31, 2023 */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|Webview history 1]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|Webview history 2]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 7, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 24|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 26, 2023===<br />
<br />
Final Projects due<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_21&diff=24416Mobile Apps 2023W Lecture 212023-03-30T03:26:48Z<p>Soma: Created page with "==Notes== <pre> March 29 -------- So I'm trying to figure out how to implement the back button properly. Currently, the back button gets triggered but doesn't work right, .goBack() isn't doing what we expected - hypothesis: state changes are updating history too often, getting copies of current page and so can't keep track of last page properly Two choices: - integrate more tightly with webview history so we can better manipulate and interact with it - ignore webv..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
March 29<br />
--------<br />
<br />
So I'm trying to figure out how to implement the back button properly.<br />
<br />
Currently, the back button gets triggered but doesn't work right, .goBack() isn't doing what we expected<br />
- hypothesis: state changes are updating history too often, getting copies of current page and so can't keep track of last page properly<br />
<br />
Two choices:<br />
- integrate more tightly with webview history so we can better manipulate and interact with it<br />
- ignore webview history, implement it all on our own and just push current page to the webview widget as needed<br />
</pre><br />
<br />
==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.webviewcompose<br />
<br />
import android.os.Bundle<br />
import android.util.Log<br />
import android.view.ViewGroup<br />
import android.webkit.WebView<br />
import android.webkit.WebViewClient<br />
import androidx.activity.ComponentActivity<br />
import androidx.activity.compose.setContent<br />
import androidx.compose.foundation.layout.Row<br />
import androidx.compose.foundation.text.ClickableText<br />
import androidx.compose.material.*<br />
import androidx.compose.runtime.*<br />
import androidx.compose.ui.graphics.Color<br />
import androidx.compose.ui.text.AnnotatedString<br />
import androidx.compose.ui.tooling.preview.Preview<br />
import androidx.compose.ui.viewinterop.AndroidView<br />
<br />
class MainActivity : ComponentActivity() {<br />
<br />
override fun onCreate(savedInstanceState: Bundle?) {<br />
super.onCreate(savedInstanceState)<br />
setContent {<br />
// Calling the composable function<br />
// to display element and its contents<br />
MainContent()<br />
}<br />
}<br />
}<br />
<br />
// Creating a composable<br />
// function to display Top Bar<br />
@Composable<br />
fun MainContent() {<br />
var curURL by remember { mutableStateOf("https://news.ycombinator.com") }<br />
var navigate by remember { mutableStateOf("") }<br />
<br />
Scaffold(<br />
// topBar = { TopAppBar(title = {Text("COMP 2601")})},<br />
topBar = {<br />
Row {<br />
ClickableText(<br />
AnnotatedString(text = "Back"),<br />
onClick = {<br />
navigate = "back"<br />
Log.d("webviewCompose", "navigate = back")<br />
}<br />
)<br />
URLBar(curURL = curURL, updateURL = { curURL = it })<br />
}<br />
},<br />
content = { MyContent(curURL = curURL, updateURL = {<br />
curURL = it<br />
Log.d("webviewCompose", "updating URL to $curURL")<br />
},<br />
navigate = navigate, resetNavigate = { navigate = ""}) }<br />
)<br />
}<br />
<br />
@Composable<br />
fun URLBar(curURL: String, updateURL: (String) -> Unit) {<br />
var newURL = ""<br />
<br />
TextField(<br />
value = curURL,<br />
//onFocusChange = { updateURL(newURL) },<br />
//onValueChange = { updatedURL: String -> newURL = updatedURL},<br />
onValueChange = updateURL,<br />
label = { Text("Enter URL here") }<br />
)<br />
}<br />
<br />
@Composable<br />
fun MyContent(curURL: String, updateURL: (String) -> Unit,<br />
navigate: String, resetNavigate: () -> Unit){<br />
<br />
class ObservableWebViewClient : WebViewClient() {<br />
<br />
override fun shouldOverrideUrlLoading(view: WebView?, url: String?): Boolean {<br />
if (url != null) {<br />
updateURL(url)<br />
}<br />
return false<br />
}<br />
}<br />
<br />
fun logWebViewHistory(view: WebView) {<br />
val history = view.copyBackForwardList()<br />
<br />
val current = history.currentIndex<br />
<br />
Log.d("webviewCompose", "Current history item: $current")<br />
<br />
Log.d("webviewCompose", "Webview BackForwardList contents:")<br />
<br />
for (i in 0..history.size -1) {<br />
val item = history.getItemAtIndex(i)<br />
<br />
if (item != null) {<br />
val u = item.originalUrl<br />
Log.d("webviewCompose", " $i: $u")<br />
}<br />
}<br />
}<br />
<br />
// Adding a WebView inside AndroidView<br />
// with layout as full screen<br />
AndroidView(factory = {<br />
WebView(it).apply {<br />
// layoutParams = ViewGroup.LayoutParams(<br />
// ViewGroup.LayoutParams.MATCH_PARENT,<br />
// ViewGroup.LayoutParams.MATCH_PARENT<br />
// )<br />
webViewClient = ObservableWebViewClient()<br />
loadUrl(curURL)<br />
}<br />
}, update = {<br />
if (navigate == "back") {<br />
Log.d("webviewCompose", "Navigating back")<br />
val currentURL = it.url<br />
Log.d("webviewCompose", "currentURL = $currentURL")<br />
it.goBack()<br />
val prevURL = it.url<br />
Log.d("webviewCompose", "prevURL = $prevURL")<br />
<br />
if (currentURL == prevURL) {<br />
Log.d("webviewCompose", "current and prev are the SAME!")<br />
}<br />
if (prevURL != null) {<br />
Log.d("webviewCompose", "updating URL after back button")<br />
updateURL(prevURL)<br />
}<br />
resetNavigate()<br />
<br />
logWebViewHistory(it)<br />
} else {<br />
Log.d("webviewCompose", "loading URL $curURL")<br />
it.loadUrl(curURL)<br />
logWebViewHistory(it)<br />
}<br />
})<br />
}<br />
<br />
// For displaying preview in<br />
// the Android Studio IDE emulator<br />
@Preview(showBackground = true)<br />
@Composable<br />
fun DefaultPreview() {<br />
MainContent()<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24415Mobile Applications (Winter 2023)2023-03-30T03:25:55Z<p>Soma: /* March 29, 2023 */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|Webview history 1]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|TBD]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 7, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 24|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 26, 2023===<br />
<br />
Final Projects due<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)_Project_Guidelines&diff=24414Mobile Applications (Winter 2023) Project Guidelines2023-03-29T18:41:14Z<p>Soma: /* Presentation */</p>
<hr />
<div>This page contains guidelines on the class project.<br />
<br />
==Project Proposal==<br />
<br />
Your project proposal should be a relatively short document describing what you plan to do for your project. You should propose to make an original app of some kind. Make sure you propose to do something that you can complete this semester. It is fine to have optional extensions that you add if you have time; the basic app, however, should be completed with some reasonable amount of functionality.<br />
<br />
Your proposal should:<br />
* Have a title and your name<br />
* If you have a collaborator, list them and explain how the collaboration will work. You can have at most one collaborator. (You both must submit separate proposals that focus on your individual responsibilities.)<br />
* Explain your motivation for what you are doing<br />
* Describe what you plan to do<br />
* Present what preliminary work you've done<br />
* Outline your development plan, including milestones and dates<br />
** Leave enough time to write your final report<br />
<br />
Your proposal will be graded based on the following criteria:<br />
* 40% Writing & presentation quality (organization, grammar, flow)<br />
* 40% Technical scope<br />
* 20% Creativity<br />
<br />
I may change the relative weights of the categories, but only if it results in a better grade (i.e., a project that is really creative but not so technically demanding might get 40% creativity, 20% technical scope).<br />
<br />
Note that part of technical scope is whether what you propose is doable within the time you have. So, if you propose something very ambitious you should have a lot of preliminary work completed.<br />
<br />
==Presentation==<br />
<br />
You will have 10 minutes to present your app to the class.<br />
<br />
In these 10 minutes, you need to do the following:<br />
* Describe at a high level what you built, and explain your motivations. Feel free to include information on your personal passions.<br />
* Describe at least three technical or design challenges that you faced and explain how each was overcome. Be specific. Explain in the context of key mobile development concepts. Consider using code excerpts.<br />
* Give your perspective on Android as a platform for implementing what you did. How did it support your work? How was it unhelpful? How did it get in the way? Make specific technical arguments please. <br />
* If you have future plans to extend or modify what you did, please outline them.<br />
<br />
Your presentation will be graded out of 10 as follows:<br />
* 2 for description & motivation<br />
* 6 for challenges<br />
* 2 for your perspective on Android<br />
Your score in each category will be based on content and style, with more of an emphasis on content. Note the scoring should give you an idea of how to allocate your presentation time.<br />
<br />
==Final Report==<br />
<br />
Your report should have the following sections:<br />
<br />
* Introduction/Overview<br />
** what did you do?<br />
* Inspirations/Background<br />
** What led to what you did?<br />
** inspiring apps, articles, daydreams, whatever<br />
** make sure to cite inspirations where feasible<br />
* Design<br />
** high-level description of what you built<br />
* Implementation<br />
** details, how it exists in Android<br />
** Explain code structure<br />
** enough detail that someone who looks at your code will have a basic idea of what is going on, how to find bits they are interested in<br />
** if chunks of your implementation came from outside sources, just give a high level summary of it and cite the source<br />
* Iteration/Evaluation<br />
** organize what you wrote in your journal, what was the process?<br />
** how close did you get to what you wanted to do?<br />
** what are the major issues remaining?<br />
** report on any user testing you did<br />
** cite resources that helped you along the way<br />
* Lessons Learned<br />
** what do you know now that you didn't know at the start of the term?<br />
* Conclusion<br />
<br />
Grading will be based on:<br />
* 30% Writing & presentation quality (organization, grammar, flow)<br />
* 30% Technical scope<br />
* 20% Creativity<br />
* 20% Evaluation/Process<br />
<br />
Again, the weighing may be changed.</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24413Mobile Applications (Winter 2023)2023-03-29T18:39:46Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|TBD]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|TBD]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 7, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 24|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 26, 2023===<br />
<br />
Final Projects due<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-29&diff=24412DistOS 2023W 2023-03-292023-03-29T18:20:38Z<p>Soma: </p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Haystack & f4<br />
-------------<br />
<br />
- What problem does Haystack solve? What problem does f4 solve?<br />
- How does Haystack work? How does f4 work?<br />
- what behavior patterns are needed to make Haystack and f4 work well?<br />
- how grounded are these patterns in human behavior?<br />
- What is the relationship between Haystack and f4?<br />
- What are the key technical insights used to build these systems?<br />
- To what degree could these systems be used for other tasks?<br />
<br />
We'll meet again at 12:18<br />
<br />
What problems do these systems solve?<br />
- high-performance, cheap photo/BLOB storage<br />
<br />
How do we make it high performance & cheap?<br />
- in part, take advantage of access patterns (long tail)<br />
- most photos won't be accessed for a long time<br />
- but recent ones will, and once accessed once they'll be accessed a lot<br />
- but people will sometimes just scroll through lots of photos too, so<br />
that should work okay<br />
<br />
Before Haystack, how did they store images?<br />
- NFS appliances (NetApp?)<br />
- CDNs<br />
<br />
What is a content delivery network (CDN)?<br />
- classically, a way for a web server to offload static assets/cache frequently accessed resources<br />
- with "edge computing" this is changing, puts dynamic data into the CDN<br />
- idea is, if large static assets are served from servers close to clients, it will be faster and will allow the primary web server & database to have less of a load and so can scale better<br />
<br />
A big problem for any CDN is where to put data?<br />
- you want the right data to be "near" the right client<br />
- but you can't store all data near every client<br />
- so, you have to predict where data is going to go<br />
- big insight behind Akamai<br />
<br />
Facebook was paying too much money using standard CDNs and it wasn't really appropriate for a lot of their workload<br />
- no good for the tail<br />
<br />
"long tail" refers to the distribution<br />
- not normal, more power law<br />
- most frequent have most of the area under the curve<br />
- but, low frequency things collectively are a huge chuck of the area<br />
<br />
<br />
Big idea of Haystack is if we can simplify metadata storage we can optimize data access times<br />
- really, want to get everything we need in one read from disk with no seeks<br />
<br />
What's the problem with deleting photos in Haystack?<br />
- marked for deletion, still available until compaction<br />
<br />
Facebook needed to fix this, wanted deletes to happen immediately<br />
<br />
How does f4 fix this?<br />
- encrypt all data<br />
- delete key to delete data<br />
<br />
So why warm storage?<br />
- different tradeoff between cost, performance, and durability<br />
<br />
Big insight is that Haystack is expensive<br />
- 3 copies of all data<br />
<br />
Can we have fewer copies but still have similar reliability guarantees?<br />
- yes, with erasure codes<br />
<br />
Erasure codes encode data in a way that you can recover lost data<br />
- robust to erasure<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Distributed_OS:_Winter_2023&diff=24411Distributed OS: Winter 20232023-03-29T15:49:38Z<p>Soma: /* April 5, 2023 */</p>
<hr />
<div>==Course Outline==<br />
<br />
[[Distributed OS: Winter 2023 Course Outline|Here]] is the course outline.<br />
<br />
==Project Help==<br />
<br />
To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.<br />
<br />
To begin selecting a paper, I suggest that you:<br />
* search on Google Scholar using keywords relating to your interests, and/or<br />
* browse the proceedings of major conferences that publish work related to distributed operating systems.<br />
<br />
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].<br />
<br />
To help you write a literature review or the background of a research paper, read the following:<br />
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]<br />
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]<br />
<br />
==Class Schedule & Readings==<br />
<br />
===[[DistOS 2023W 2023-01-09|January 9, 2023]]===<br />
<br />
Introduction<br />
<br />
===[[DistOS 2023W 2023-01-11|January 11, 2023]]===<br />
<br />
Designing a distributed operating system<br />
<br />
===[[DistOS 2023W 2023-01-16|January 16, 2023]]===<br />
<br />
The Early Internet & Multics:<br />
* [https://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]<br />
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video<br />
Optional: Browse around [https://www.multicians.org/ the Multicians website].<br />
<br />
===[[DistOS 2023W 2023-01-18|January 18, 2023]]===<br />
<br />
UNIX<br />
* [https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/ Richard Jensen, "Unix at 50: How the OS that powered smartphones started from failure" (2019, Ars Technica)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]<br />
* [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video<br />
Note that the video covers the main points of the UNIX paper.<br />
<br />
===[[DistOS 2023W 2023-01-23|January 23, 2023]]===<br />
<br />
The Mother of All Demos:<br />
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].<br />
* [https://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]<br />
<br />
The Alto:<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])<br />
<br />
===[[DistOS 2023W 2023-01-25|January 25, 2023]]===<br />
<br />
Distributed Shared Memory<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/kai-li-shared.pdf Kai Li, "Shared virtual memory on loosely coupled multiprocessors" (1986)] '''(OPTIONAL)'''<br />
<br />
===[[DistOS 2023W 2023-01-30|January 30, 2023]]===<br />
<br />
Remote Procedure Calls<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/nelson1981-rpc.pdf Bruce J. Nelson, "Remote Procedure Call" (1981)] (Read the summary starting on p. 224, look at the Intro and Background if you have time.)<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/rfc1050.txt Sun Microsystems, "RPC: Remote Procedure Call Protocol Specification" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-01|February 1, 2023]]===<br />
<br />
LOCUS & NFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]<br />
<br />
===[[DistOS 2023W 2023-02-06|February 6, 2023]]===<br />
<br />
Sprite, AFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-08|February 8, 2023]]===<br />
<br />
Plan 9 & Inferno<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2018f/doward97-inferno.pdf Doward et al., The Inferno Operating System (1997)]<br />
* [http://www.inferno-os.info/inferno/ Inferno OS website] (browse)<br />
* [http://doc.cat-v.org/inferno/4th_edition/styx Rob Pike and Dennis M. Ritchie, "The Styx Architecture for Distributed Systems"]<br />
<br />
===[[DistOS 2023W 2023-02-13|February 13, 2023]]===<br />
<br />
Midterm review<br />
<br />
===February 15, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/~soma/distos/2023w/comp4000-2023w-midterm.pdf Midterm exam]<br />
<br />
===[[DistOS 2023W 2023-02-27|February 27, 2023]]===<br />
<br />
"Web Scale" discussion<br />
<br />
===[[DistOS 2023W 2023-03-01|March 1, 2023]]===<br />
<br />
OceanStore & BOINC<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]<br />
<br />
Project Proposal due March 3rd<br />
<br />
===[[DistOS 2023W 2023-03-06|March 6, 2023]]===<br />
<br />
NASD & Tapestry<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]<br />
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]<br />
<br />
Background (optional but helpful):<br />
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]<br />
<br />
===[[DistOS 2023W 2023-03-08|March 8, 2023]]===<br />
<br />
Chubby & GFS<br />
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]<br />
* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]<br />
<br />
===[[DistOS 2023W 2023-03-13|March 13, 2023]]===<br />
<br />
Ceph<br />
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]<br />
<br />
===[[DistOS 2023W 2023-03-15|March 15, 2023]]===<br />
<br />
BigTable & MapReduce<br />
* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]<br />
* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)<br />
<br />
===[[DistOS 2023W 2023-03-20|March 20, 2023]]===<br />
<br />
Omega & Borg<br />
* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]<br />
* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]<br />
<br />
===[[DistOS 2023W 2023-03-22|March 22, 2023]]===<br />
<br />
Cassandra & Dynamo<br />
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]<br />
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]<br />
<br />
===[[DistOS 2023W 2023-03-27|March 27, 2023]]===<br />
<br />
Zookeeper & Delos<br />
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]<br />
* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]<br />
<br />
===[[DistOS 2023W 2023-03-29|March 29, 2023]]===<br />
<br />
Haystack & F4<br />
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]<br />
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]<br />
<br />
===[[DistOS 2023W 2023-04-03|April 3, 2023]]===<br />
<br />
Spanner & Tensorflow<br />
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]<br />
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]<br />
<br />
===[[DistOS 2023W 2023-04-05|April 5, 2023]]===<br />
<br />
Class wrap up, final exam review<br />
<br />
===April 10, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-29&diff=24410DistOS 2023W 2023-03-292023-03-29T15:48:40Z<p>Soma: Created page with "==Notes== <pre> Haystack & f4 ------------- - What problem does Haystack solve? What problem does f4 solve? - How does Haystack work? How does f4 work? - what behavior patterns are needed to make Haystack and f4 work well? - how grounded are these patterns in human behavior? - What is the relationship between Haystack and f4? - What are the key technical insights used to build these systems? - To what degree could these systems be used for other tasks? We'l..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Haystack & f4<br />
-------------<br />
<br />
- What problem does Haystack solve? What problem does f4 solve?<br />
- How does Haystack work? How does f4 work?<br />
- what behavior patterns are needed to make Haystack and f4 work well?<br />
- how grounded are these patterns in human behavior?<br />
- What is the relationship between Haystack and f4?<br />
- What are the key technical insights used to build these systems?<br />
- To what degree could these systems be used for other tasks?<br />
<br />
We'll meet again at 12:18<br />
<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-27&diff=24409DistOS 2023W 2023-03-272023-03-27T18:17:59Z<p>Soma: </p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Zookeeper & Delos<br />
-----------------<br />
<br />
Consensus<br />
- why is this such an important problem in distributed OS?<br />
<br />
Notice that consensus isn't a concern in single-system operating systems<br />
- we get "consensus" simply by having one copy of data<br />
<br />
But in a distributed system we have copies of data & distributed state<br />
- so if we want all of the system to be in the "same" state, we need consensus<br />
- consensus has to be constructed, it isn't the natural state of the system<br />
<br />
Consensus in natural systems can take interesting forms, not really like how we currently do things in computers<br />
- consider flocking behavior<br />
<br />
Artificial systems typically need greater precision in their consensus, and so they need different solutions than natural systems<br />
<br />
In the papers, don't focus on the algorithms<br />
- instead focus on the APIs, abstractions<br />
- how do they compare?<br />
- compare "file-like" abstractions to "log-like" abstractions<br />
<br />
What is the real difference between a log and a file?<br />
<br />
Groups go until 12:20<br />
<br />
AFTER GROUPS<br />
<br />
What do the APIs look for the different systems?<br />
<br />
Chubby: whole-file API with exclusive access (locking)<br />
Zookeeper: whole-file API without locking<br />
Delos: shared log API<br />
<br />
Why does Zookeeper avoid locking?<br />
- slow clients can block everyone else<br />
<br />
Wait-free access vs blocking<br />
<br />
With chubby, a client can try to do something and will be blocked if the resource is taken<br />
<br />
With zookeeper, operations can always work, so if you want locking mechanisms you have to build on top. Only guarantee is there is a fixed, universal order to writes, and when you read you get the same data.<br />
<br />
Key addition: watching nodes<br />
<br />
With blocking, you are "notified" when the resource is available, you just can't do anything in the meantime<br />
<br />
With "watching", you get a callback when state has changed, and you can work in the meantime on other things<br />
<br />
Which do you think is easier to use, Zookeeper or Chubby?<br />
- chubby is easier, because blocking semantics are normally easier to reason about, that's why we traditionally use locks for so many concurrency problems<br />
- zookeeper is lower level in a sense, you have to build the abstraction you want on top of it (and potentially can get it wrong)<br />
- locks are built on top of zookeeper, are not a fundamental part of it<br />
- but zookeeper gets more flexibility, allows for greater optimization based on application needs<br />
<br />
So what is Delos?<br />
- going lower level than zookeeper, to the fundamentals of what is needed for consensus<br />
- abstraction is appending to, sealing, and checking the end of a log file<br />
(Loglet)<br />
<br />
Why is a log more fundamental than a file for consensus?<br />
- keeps track of time<br />
- keeps track of *state changes*<br />
<br />
With a file, you only get the most recent state, you can't see the history.<br />
<br />
You can implement files on top of logs<br />
- that's a log-structured filesystem<br />
<br />
Fundamental nature of consensus is an agreement over the the time of events<br />
- and that's what hard in a distributed system, every view of events is relative<br />
- if you make a local change to state and hear about a remote change of state, did your local change happen "before" or "after" the remote change?<br />
- really matters if you both modified the same thing or related things<br />
<br />
If you have a globally consistent view of events, this never happens<br />
- you always know who did what when relative to the other<br />
- so if there is a conflict, you know who was first or last and can resolve it<br />
<br />
A file is just the result of a sequence of state updates<br />
- so if you know the sequence of updates, you know the contents of the file<br />
<br />
So why bother with focusing on logs?<br />
- because we can build various consensus algorithms on top of it, clients just see a log, and different ways of building the log will have different performance characteristics<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-27&diff=24408DistOS 2023W 2023-03-272023-03-27T15:52:02Z<p>Soma: Created page with "==Notes== <pre> Zookeeper & Delos ----------------- Consensus - why is this such an important problem in distributed OS? Notice that consensus isn't a concern in single-system operating systems - we get "consensus" simply by having one copy of data But in a distributed system we have copies of data & distributed state - so if we want all of the system to be in the "same" state, we need consensus - consensus has to be constructed, it isn't the natural state of the..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Zookeeper & Delos<br />
-----------------<br />
<br />
Consensus<br />
- why is this such an important problem in distributed OS?<br />
<br />
Notice that consensus isn't a concern in single-system operating systems<br />
- we get "consensus" simply by having one copy of data<br />
<br />
But in a distributed system we have copies of data & distributed state<br />
- so if we want all of the system to be in the "same" state, we need consensus<br />
- consensus has to be constructed, it isn't the natural state of the system<br />
<br />
Consensus in natural systems can take interesting forms, not really like how we currently do things in computers<br />
- consider flocking behavior<br />
<br />
Artificial systems typically need greater precision in their consensus, and so they need different solutions than natural systems<br />
<br />
In the papers, don't focus on the algorithms<br />
- instead focus on the APIs, abstractions<br />
- how do they compare?<br />
- compare "file-like" abstractions to "log-like" abstractions<br />
<br />
What is the real difference between a log and a file?<br />
<br />
Groups go until 12:20<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_20&diff=24407Mobile Apps 2023W Lecture 202023-03-24T21:15:39Z<p>Soma: Created page with "==Code== <syntaxhighlight lang="kotlin" line> package carleton.comp2601.webviewcompose import android.os.Bundle import android.util.Log import android.view.ViewGroup import android.webkit.WebView import android.webkit.WebViewClient import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.compose.foundation.layout.Row import androidx.compose.foundation.text.ClickableText import androidx.compose.material.* import androidx.com..."</p>
<hr />
<div>==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.webviewcompose<br />
<br />
import android.os.Bundle<br />
import android.util.Log<br />
import android.view.ViewGroup<br />
import android.webkit.WebView<br />
import android.webkit.WebViewClient<br />
import androidx.activity.ComponentActivity<br />
import androidx.activity.compose.setContent<br />
import androidx.compose.foundation.layout.Row<br />
import androidx.compose.foundation.text.ClickableText<br />
import androidx.compose.material.*<br />
import androidx.compose.runtime.*<br />
import androidx.compose.ui.graphics.Color<br />
import androidx.compose.ui.text.AnnotatedString<br />
import androidx.compose.ui.tooling.preview.Preview<br />
import androidx.compose.ui.viewinterop.AndroidView<br />
<br />
class MainActivity : ComponentActivity() {<br />
<br />
override fun onCreate(savedInstanceState: Bundle?) {<br />
super.onCreate(savedInstanceState)<br />
setContent {<br />
// Calling the composable function<br />
// to display element and its contents<br />
MainContent()<br />
}<br />
}<br />
}<br />
<br />
// Creating a composable<br />
// function to display Top Bar<br />
@Composable<br />
fun MainContent() {<br />
var curURL by remember { mutableStateOf("https://news.ycombinator.com") }<br />
var navigate by remember { mutableStateOf("") }<br />
<br />
Scaffold(<br />
// topBar = { TopAppBar(title = {Text("COMP 2601")})},<br />
topBar = {<br />
Row {<br />
ClickableText(<br />
AnnotatedString(text = "Back"),<br />
onClick = {<br />
navigate = "back"<br />
Log.d("webviewCompose", "navigate = back")<br />
}<br />
)<br />
URLBar(curURL = curURL, updateURL = { curURL = it })<br />
}<br />
},<br />
content = { MyContent(curURL = curURL, updateURL = { curURL = it },<br />
navigate = navigate, resetNavigate = { navigate = ""}) }<br />
)<br />
}<br />
<br />
@Composable<br />
fun URLBar(curURL: String, updateURL: (String) -> Unit) {<br />
var newURL = ""<br />
<br />
TextField(<br />
value = curURL,<br />
//onFocusChange = { updateURL(newURL) },<br />
//onValueChange = { updatedURL: String -> newURL = updatedURL},<br />
onValueChange = updateURL,<br />
label = { Text("Enter URL here") }<br />
)<br />
}<br />
<br />
@Composable<br />
fun MyContent(curURL: String, updateURL: (String) -> Unit,<br />
navigate: String, resetNavigate: () -> Unit){<br />
<br />
class ObservableWebViewClient : WebViewClient() {<br />
<br />
override fun shouldOverrideUrlLoading(view: WebView?, url: String?): Boolean {<br />
if (url != null) {<br />
updateURL(url)<br />
}<br />
return false<br />
}<br />
}<br />
<br />
// Adding a WebView inside AndroidView<br />
// with layout as full screen<br />
AndroidView(factory = {<br />
WebView(it).apply {<br />
// layoutParams = ViewGroup.LayoutParams(<br />
// ViewGroup.LayoutParams.MATCH_PARENT,<br />
// ViewGroup.LayoutParams.MATCH_PARENT<br />
// )<br />
webViewClient = ObservableWebViewClient()<br />
loadUrl(curURL)<br />
}<br />
}, update = {<br />
if (navigate == "back") {<br />
Log.d("webviewCompose", "Navigating back")<br />
it.goBack()<br />
val theURL = it.url<br />
if (theURL != null) {<br />
updateURL(theURL)<br />
}<br />
resetNavigate()<br />
} else {<br />
it.loadUrl(curURL)<br />
}<br />
})<br />
}<br />
<br />
// For displaying preview in<br />
// the Android Studio IDE emulator<br />
@Preview(showBackground = true)<br />
@Composable<br />
fun DefaultPreview() {<br />
MainContent()<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24406Mobile Applications (Winter 2023)2023-03-24T21:14:26Z<p>Soma: /* March 24, 2023 */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|Webview URL bar 2]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|TBD]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|TBD]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 7, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 24|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_19&diff=24405Mobile Apps 2023W Lecture 192023-03-22T21:13:12Z<p>Soma: Created page with "==Notes== <pre> March 22 -------- Integrating webview with HNTree we have composable functions that we can copy over, but how should the app actually work? Should start up as a web browser with a URL bar, with a button to the side of the URL bar that says "History". Clicking on the button will switch the view to the History view. If the user taps on a specific past URL, visit the page (potentially with confirmation) What's the strategy? - first, change the webvie..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
March 22<br />
--------<br />
<br />
Integrating webview with HNTree<br />
<br />
we have composable functions that we can copy over, but how should the app actually work?<br />
<br />
Should start up as a web browser with a URL bar, with a button to the side of the URL bar that says "History". Clicking on the button will switch the view to the History view. If the user taps on a specific past URL, visit the page (potentially with confirmation)<br />
<br />
What's the strategy?<br />
- first, change the webviewCompose app to have a URL bar and a history button<br />
- then, copy code over to historyviewer<br />
<br />
Why?<br />
- separate complexity of interface design from "backend" (core functionality) design<br />
<br />
<br />
My general strategy - get to "working code" ASAP, never have something that doesn't run for too long<br />
- if you don't do this, conceptual issues can persist for way too long<br />
</pre><br />
<br />
==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.webviewcompose<br />
<br />
import android.os.Bundle<br />
import android.view.ViewGroup<br />
import android.webkit.WebView<br />
import android.webkit.WebViewClient<br />
import androidx.activity.ComponentActivity<br />
import androidx.activity.compose.setContent<br />
import androidx.compose.material.*<br />
import androidx.compose.runtime.*<br />
import androidx.compose.ui.graphics.Color<br />
import androidx.compose.ui.tooling.preview.Preview<br />
import androidx.compose.ui.viewinterop.AndroidView<br />
<br />
class MainActivity : ComponentActivity() {<br />
<br />
override fun onCreate(savedInstanceState: Bundle?) {<br />
super.onCreate(savedInstanceState)<br />
setContent {<br />
// Calling the composable function<br />
// to display element and its contents<br />
MainContent()<br />
}<br />
}<br />
}<br />
<br />
// Creating a composable<br />
// function to display Top Bar<br />
@Composable<br />
fun MainContent() {<br />
var curURL by remember { mutableStateOf("https://news.ycombinator.com") }<br />
<br />
Scaffold(<br />
// topBar = { TopAppBar(title = {Text("COMP 2601")})},<br />
topBar = { URLBar(curURL = curURL, updateURL = { curURL = it }) },<br />
content = { MyContent(curURL = curURL, updateURL = { curURL = it }) }<br />
)<br />
}<br />
<br />
@Composable<br />
fun URLBar(curURL: String, updateURL: (String) -> Unit) {<br />
var newURL = ""<br />
<br />
TextField(<br />
value = curURL,<br />
//onFocusChange = { updateURL(newURL) },<br />
//onValueChange = { updatedURL: String -> newURL = updatedURL},<br />
onValueChange = updateURL,<br />
label = { Text("Enter URL here") }<br />
)<br />
}<br />
<br />
// Creating a composable<br />
// function to create WebView<br />
// Calling this function as<br />
// content in the above function<br />
@Composable<br />
fun MyContent(curURL: String, updateURL: (String) -> Unit){<br />
<br />
<br />
// Adding a WebView inside AndroidView<br />
// with layout as full screen<br />
AndroidView(factory = {<br />
WebView(it).apply {<br />
// layoutParams = ViewGroup.LayoutParams(<br />
// ViewGroup.LayoutParams.MATCH_PARENT,<br />
// ViewGroup.LayoutParams.MATCH_PARENT<br />
// )<br />
webViewClient = WebViewClient()<br />
loadUrl(curURL)<br />
}<br />
}, update = {<br />
it.loadUrl(curURL)<br />
})<br />
}<br />
<br />
// For displaying preview in<br />
// the Android Studio IDE emulator<br />
@Preview(showBackground = true)<br />
@Composable<br />
fun DefaultPreview() {<br />
MainContent()<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24404Mobile Applications (Winter 2023)2023-03-22T21:11:44Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
===March 22, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 19|Webview URL bar 1]]<br />
<br />
===March 24, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 20|TBD]]<br />
<br />
===March 29, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 21|TBD]]<br />
<br />
===March 31, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 22|TBD]]<br />
<br />
===April 5, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 23|TBD]]<br />
<br />
===April 7, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 24|TBD]]<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-22&diff=24403DistOS 2023W 2023-03-222023-03-22T17:32:28Z<p>Soma: </p>
<hr />
<div>==Discussion Questions==<br />
<br />
* What problems are Cassandra and Dynamo built to solve? How do these problems inform their design?<br />
* What are the key technical insights or algorithms behind Cassandra and Dynamo?<br />
* What infrastructure do Cassandra and Dynamo seem to rely on? How does this compare with the systems made by Google?<br />
<br />
==Notes==<br />
<br />
<pre><br />
March 22<br />
--------<br />
<br />
Ploblems designed to solve<br />
* Cassandra: inbox search<br />
* Dynamo: shopping carts & user sessions<br />
<br />
Infrastructure<br />
* Cassandra: Ganglia (perf monitoring), Zookeeper (consistent state), MapReduce, MySQL<br />
* Dynamo: local node storage (MySQL, Berkeley DB, others)<br />
<br />
Why does Google rely so much on a distributed filesystem while Facebook and Amazon don't?<br />
- Facebook and Amazon just had to deal with user/merchant data<br />
- Google crawled the web, indexed it<br />
<br />
Amazon is different from the other two in that it uses a service-oriented architecture<br />
- focus on decentralization, connection via well-defined APIs<br />
- versus more monolithic systems with lots of interdependencies<br />
- Google uses one single source code repository that everyone can access!<br />
<br />
The real difference is about trust<br />
- monolithic but distributed system: everything is assumed to work together<br />
- SOA: everyone else may abuse your system, you must design around that<br />
<br />
The difference is between implicit vs explicit guarantees of functionality<br />
<br />
"two pizza rule" for Amazon?<br />
- organized into small teams<br />
<br />
Every distributed system has to be built around smaller parts, the question is how trusted each of these parts are by other parts of the system<br />
<br />
Amazon was uniquely suited to build AWS, because its services were mostly already ready for "outsiders" to use<br />
<br />
Both systems make use of gossip protocols<br />
- scuttlebutt for Cassandra<br />
<br />
What is a gossip protocol?<br />
- sharing state by "talking" with peers<br />
- should have some guarantees that everyone learns what they need to know<br />
- gossip can include local and global data<br />
<br />
Dynamo has a weird approach to consistency issues<br />
- most systems resolve consistency when writing data, or after<br />
data has been written and before reading<br />
- dynamo resolves while reading<br />
<br />
If you do consistency on writes, you can slow down writes<br />
By moving conflict resolution to reads, you ensure writes happen with minimal delay (maximizing performance and durability)<br />
<br />
Dynamo's strategy makes sense when you realize their focus is on shopping carts<br />
- better to have more items in the cart than fewer!<br />
- definitely a domain-specific optimization<br />
<br />
Cassandra is focused on storing inboxes, enabling search<br />
- so they also have to prioritize writes, but want to keep it more consistent than amazon<br />
- focus is more on making the inbox responsive so users don't get annoyed<br />
(don't want them leaving facebook after all)<br />
- prioritize newer messages over older messages, but keep all the data<br />
<br />
Note Cassandra's local storage looks like a log-structured filesystem<br />
<br />
<br />
Traditional filesystems have metadata and data all intermixed<br />
- wide trees of blocks for storing data, lots of pointers (block references)<br />
- poor locality, may require many seeks to get file contents<br />
(read inode, read indirect blocks, read data blocks)<br />
- extents only partially solve this<br />
<br />
The big problem with traditional filesystems is with writes<br />
- have to write to random parts of the disk, and random writes are *much* slower than sequential writes<br />
- and if data isn't written to disk, you'll lose it<br />
<br />
Idea: write data in a sequential log first, then write it to the tree<br />
- journaled filesystems do this, but typically just journal metadata, not data<br />
- journaled fs make fsck MUCH faster<br />
<br />
log structured filesystem gets rid of the regular tree, puts everything in the journal (the log)<br />
- add in in-memory indexes to track what is current<br />
- add compaction to remove old versions on disk and free up space<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-22&diff=24402DistOS 2023W 2023-03-222023-03-22T15:33:58Z<p>Soma: Created page with "==Discussion Questions== * What problems are Cassandra and Dynamo built to solve? How do these problems inform their design? * What are the key technical insights or algorithms behind Cassandra and Dynamo? * What infrastructure do Cassandra and Dynamo seem to rely on? How does this compare with the systems made by Google?"</p>
<hr />
<div>==Discussion Questions==<br />
<br />
* What problems are Cassandra and Dynamo built to solve? How do these problems inform their design?<br />
* What are the key technical insights or algorithms behind Cassandra and Dynamo?<br />
* What infrastructure do Cassandra and Dynamo seem to rely on? How does this compare with the systems made by Google?</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-20&diff=24400DistOS 2023W 2023-03-202023-03-21T03:46:58Z<p>Soma: Created page with "==Notes== <pre> March 20 -------- Project Proposals - if you want to re-submit, you can, and I will replace the grade with your new one - Need to finish it by March 27th - some proposals were just way too broad - need to narrow it down - some were too focused on one paper - needs to cover multiple papers - some tried to argue for a technical point - you aren't doing original research - focus on what the papers say, have a thesis about patterns..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
March 20<br />
--------<br />
<br />
Project Proposals<br />
- if you want to re-submit, you can, and I will replace the grade with your new one<br />
- Need to finish it by March 27th<br />
<br />
- some proposals were just way too broad<br />
- need to narrow it down<br />
- some were too focused on one paper<br />
- needs to cover multiple papers<br />
- some tried to argue for a technical point<br />
- you aren't doing original research<br />
- focus on what the papers say, have a thesis about patterns<br />
in their arguments<br />
<br />
Remember the thesis comes after you've read and understood the papers<br />
- represents some of the insight you got from the papers<br />
<br />
Borg & Omega<br />
<br />
A container<br />
- packaged Unix-like userland<br />
- generally to be run on a Linux kernel<br />
- only dependency is on the Linux kernel, all<br />
userspace dependencies are included<br />
- popular in the cloud space because allows for composition of workloads on a host<br />
- can run without seeing each other with OS virtualization<br />
(private filesystem, user ID and process IDs, resource quotas, etc)<br />
- note that containers don't *really* contain from a security perspective<br />
- must use advanced OS security mechanisms (SELinux, system call filtering)<br />
to keep them separate, even then generally not enough<br />
- if running untrusted workloads, generally use hardware virtual machines to separate<br />
- area of my research how to do better! (look up bpfcontain, bpfbox)<br />
- remember, virtualization is not confinement!<br />
(virtualization is like putting on VR goggles, doesn't change how<br />
you can affect the world just what you see)<br />
<br />
<br />
Remember with Google (Borg, Omega), it is running containers that are trusted (i.e., they made them all), so they don't have to worry about containers going bad except if they are compromised, and they have other mechanisms to deal with that<br />
<br />
AFTER GROUP DISCUSSION<br />
<br />
What is Borg?<br />
- container orchestration<br />
- foundation for Google's distributed OS<br />
- unit of resource management is the container, not the process<br />
- notice the fate of UNIX, it isn't the operating system,<br />
it became the new process (rather than make a distributed UNIX)<br />
<br />
So they had Borg, why did they make Kubernetes?<br />
<br />
How did they make Borg?<br />
- Borg is old and dates from the early days of Google<br />
- didn't rely on hardware virtualization since their systems didn't support it!<br />
- so it is full of design choices that made sense at the time, but maybe don't now<br />
- for example, only one IP address per host<br />
- Borg was made specifically for Google, co-evolved with it<br />
- doesn't really make sense outside of their environment<br />
<br />
<br />
Kubernetes is the Google engineers getting a second chance to build Borg "right"<br />
- what they want to use<br />
- but really, it is a bit *too* general, can handle<br />
Google-scale workloads but very few need that much scalability<br />
and the cost is complexity<br />
<br />
What is Omega?<br />
- research project to do better scheduling than Borg<br />
- I'm sure many of the ideas have been put into Borg by now<br />
<br />
Scheduling is a classic OS problem<br />
- lots of theoretical result<br />
- but in practice schedulers are quite messy<br />
- need domain-specific knowledge to decide what is important and<br />
to understand its needs<br />
- will tend to have heterogeneous workloads<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_18&diff=24399Mobile Apps 2023W Lecture 182023-03-17T20:52:12Z<p>Soma: Created page with "==Code== ===MainActivity.kt=== <syntaxhighlight lang="kotlin" line> package carleton.comp2601.historyviewer import android.net.Uri import android.os.Bundle import android.util.Log import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.lazy.LazyColumn import androidx.compose.material.MaterialTheme import androidx.compose.material.Surface import andro..."</p>
<hr />
<div>==Code==<br />
<br />
===MainActivity.kt===<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.historyviewer<br />
<br />
import android.net.Uri<br />
import android.os.Bundle<br />
import android.util.Log<br />
import androidx.activity.ComponentActivity<br />
import androidx.activity.compose.setContent<br />
import androidx.compose.foundation.layout.fillMaxSize<br />
import androidx.compose.foundation.lazy.LazyColumn<br />
import androidx.compose.material.MaterialTheme<br />
import androidx.compose.material.Surface<br />
import androidx.compose.material.Text<br />
import androidx.compose.runtime.Composable<br />
import androidx.compose.ui.Modifier<br />
import androidx.compose.ui.tooling.preview.Preview<br />
import carleton.comp2601.historyviewer.ui.theme.HistoryViewerTheme<br />
import androidx.compose.foundation.lazy.items<br />
import androidx.compose.foundation.text.ClickableText<br />
import androidx.compose.runtime.mutableStateListOf<br />
import androidx.compose.runtime.remember<br />
import androidx.compose.ui.text.AnnotatedString<br />
<br />
class MainActivity : ComponentActivity() {<br />
override fun onCreate(savedInstanceState: Bundle?) {<br />
super.onCreate(savedInstanceState)<br />
<br />
var hnTreeRoot = HNTree()<br />
hnTreeRoot.add("www.carleton.ca")<br />
hnTreeRoot.add("www.google.com")<br />
hnTreeRoot.add("www.cbc.ca.")<br />
hnTreeRoot.add("www.mit.edu")<br />
hnTreeRoot.add("www.google.com")<br />
<br />
<br />
setContent {<br />
HistoryViewerTheme {<br />
// A surface container using the 'background' color from the theme<br />
Surface(<br />
modifier = Modifier.fillMaxSize(),<br />
color = MaterialTheme.colors.background<br />
) {<br />
Start(hnTreeRoot)<br />
}<br />
}<br />
}<br />
}<br />
}<br />
<br />
@Composable<br />
fun Start(hnTreeRoot: HNTree) {<br />
<br />
val allNodes = remember {<br />
mutableStateListOf<HNTree>()<br />
}<br />
<br />
hnTreeRoot.addMyNodes(allNodes)<br />
<br />
LazyColumn {<br />
item {<br />
Text(text = "HOSTS:")<br />
}<br />
<br />
items(allNodes) { child -><br />
val spacing = " ".repeat(child.treeLevel)<br />
val nodeName = child.hostSuffix<br />
ClickableText(text = AnnotatedString("$spacing$nodeName"), onClick = {<br />
child.toggleExpanded()<br />
allNodes.clear()<br />
hnTreeRoot.addMyNodes(allNodes)<br />
Log.d("HistoryViewer","toggled node on $nodeName")<br />
})<br />
}<br />
}<br />
}<br />
<br />
</syntaxhighlight><br />
<br />
===HNTree.kt===<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.historyviewer<br />
<br />
import android.util.Log<br />
<br />
class HNTree {<br />
var hostComponent: String = ""<br />
var hostSuffix: String = ""<br />
var parent: HNTree = this<br />
var children: MutableList<HNTree> = mutableListOf()<br />
var treeLevel: Int = 0<br />
var expanded = true<br />
<br />
fun add(hostPrefix: String) {<br />
var allComponents: List<String><br />
var component: String<br />
var child: HNTree? = null<br />
<br />
/* still have to deal with getting to the end */<br />
<br />
allComponents = hostPrefix.split(".")<br />
if (allComponents.last() == "") {<br />
allComponents = allComponents.dropLast(1)<br />
}<br />
component = allComponents.last()<br />
<br />
for (c in this.children) {<br />
if (c.hostComponent == component) {<br />
child = c<br />
break<br />
}<br />
}<br />
<br />
if (child == null) {<br />
child = HNTree()<br />
child.parent = this<br />
child.hostComponent = component<br />
child.hostSuffix = "$component.$hostSuffix"<br />
child.treeLevel = this.treeLevel + 1<br />
children.add(child)<br />
}<br />
<br />
val hostRemainder = allComponents.dropLast(1).joinToString(".")<br />
<br />
Log.d("HNTree", "$hostSuffix adding child $hostRemainder")<br />
if (component != hostPrefix) {<br />
child.add(hostRemainder)<br />
}<br />
}<br />
<br />
fun addMyNodes(nodes: MutableList<HNTree>) {<br />
nodes.add(this)<br />
if (expanded) {<br />
//Log.d("HNTree", "$hostSuffix is expanded " )<br />
for (n in children) {<br />
n.addMyNodes(nodes)<br />
}<br />
}<br />
//Log.d("HNTree", "$hostSuffix added " + nodes.count() + " nodes." )<br />
}<br />
<br />
fun toggleExpanded() {<br />
expanded = !expanded<br />
}<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24398Mobile Applications (Winter 2023)2023-03-17T20:50:09Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
===March 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 18|Browser History 5]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_17&diff=24397Mobile Apps 2023W Lecture 172023-03-15T20:00:25Z<p>Soma: Created page with "==Code== <syntaxhighlight lang="kotlin" line> package carleton.comp2601.historyviewer class HNTree { var hostComponent: String = "" var hostSuffix: String = "" var parent: HNTree = this var children: MutableList<HNTree> = mutableListOf() var treeLevel: Int = 0 var expanded: true fun add(hostPrefix: String) { var allComponents: List<String> var component: String var child: HNTree? = null /* still have to deal..."</p>
<hr />
<div>==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.historyviewer<br />
<br />
class HNTree {<br />
var hostComponent: String = ""<br />
var hostSuffix: String = ""<br />
var parent: HNTree = this<br />
var children: MutableList<HNTree> = mutableListOf()<br />
var treeLevel: Int = 0<br />
var expanded: true<br />
<br />
fun add(hostPrefix: String) {<br />
var allComponents: List<String><br />
var component: String<br />
var child: HNTree? = null<br />
<br />
/* still have to deal with getting to the end */<br />
<br />
allComponents = hostPrefix.split(".")<br />
if (allComponents.last() == "") {<br />
allComponents = allComponents.dropLast(1)<br />
}<br />
component = allComponents.last()<br />
<br />
if (component == hostPrefix) {<br />
return<br />
}<br />
<br />
for (c in this.children) {<br />
if (c.hostComponent == component) {<br />
child = c<br />
break<br />
}<br />
}<br />
<br />
if (child == null) {<br />
child = HNTree()<br />
child.parent = this<br />
child.hostComponent = component<br />
child.hostSuffix = "$component.$hostSuffix"<br />
child.treeLevel++<br />
children.add(child)<br />
}<br />
<br />
child.add(allComponents.dropLast(1).joinToString())<br />
}<br />
<br />
fun nextNodes(n: Int): List<HNTree> {<br />
for (i in 0..n -1) {<br />
<br />
}<br />
}<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24396Mobile Applications (Winter 2023)2023-03-15T19:59:43Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
===March 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 17|Browser History 4]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-15&diff=24395DistOS 2023W 2023-03-152023-03-15T16:56:40Z<p>Soma: </p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Bigtable & MapReduce<br />
--------------------<br />
<br />
When you think about BigTable, focus on figure 1 (to understand what it is doing) and Figure 4 (to understand how).<br />
<br />
Remember that GFS requires structured information to be stored (because data can be duplicated), BigTable is one of the ways GFS files can be organized<br />
<br />
To what extent is BigTable a database?<br />
<br />
<br />
For MapReduce, think about the kind of tasks Google wanted to perform on its web crawls<br />
- generating an index, for example<br />
- gather statistics<br />
<br />
Consider the complexity of tasks that you could do with map, and you could do with reduce, noting that map is embarassingly parallel and reduce isn't.<br />
<br />
Answer in group<br />
- To what extent is BigTable a database?<br />
- how does the design of GFS influence the implementation of BigTable<br />
- what problems can be solved with MapReduce? What problems can't (efficiently)?<br />
<br />
AFTER GROUP DISCUSSIONS<br />
- BigTable is a database, but not a relational one<br />
- really, a structured key-value store<br />
<br />
Google builds infrastructure by combining systems, building on top of what they have<br />
- very little open sourced<br />
- but duplicates have been created as open source, this is much of Hadoop<br />
<br />
BigTable & GFS<br />
- note how BigTable is oriented around immutability and appends<br />
- not how standard databases are built!<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-15&diff=24394DistOS 2023W 2023-03-152023-03-15T15:44:43Z<p>Soma: Created page with "==Notes== <pre> Bigtable & MapReduce -------------------- When you think about BigTable, focus on figure 1 (to understand what it is doing) and Figure 4 (to understand how). Remember that GFS requires structured information to be stored (because data can be duplicated), BigTable is one of the ways GFS files can be organized To what extent is BigTable a database? For MapReduce, think about the kind of tasks Google wanted to perform on its web crawls - generating an..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Bigtable & MapReduce<br />
--------------------<br />
<br />
When you think about BigTable, focus on figure 1 (to understand what it is doing) and Figure 4 (to understand how).<br />
<br />
Remember that GFS requires structured information to be stored (because data can be duplicated), BigTable is one of the ways GFS files can be organized<br />
<br />
To what extent is BigTable a database?<br />
<br />
<br />
For MapReduce, think about the kind of tasks Google wanted to perform on its web crawls<br />
- generating an index, for example<br />
- gather statistics<br />
<br />
Consider the complexity of tasks that you could do with map, and you could do with reduce, noting that map is embarassingly parallel and reduce isn't.<br />
<br />
Answer in group<br />
- To what extent is BigTable a database?<br />
- how does the design of GFS influence the implementation of BigTable<br />
- what problems can be solved with MapReduce? What problems can't (efficiently)?<br />
<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-13&diff=24389DistOS 2023W 2023-03-132023-03-13T18:27:19Z<p>Soma: </p>
<hr />
<div>==Discussion Questions==<br />
<br />
* List all the terms and acronyms in the Ceph paper and discus their meaning and relationship with each other.<br />
* To what degree is Ceph POSIX compliant? Is there a cost for this?<br />
* Discuss Figures 1-3 in Crush, what does each say?<br />
<br />
==Notes==<br />
<br />
<pre><br />
Ceph & CRUSH<br />
------------<br />
<br />
The big insight of this work is that there are different approaches to managing metadata in distributed filesystems<br />
<br />
Is metadata a problem in single-device filesystems?<br />
<br />
In UNIX, there are three timestamps associated with every inode<br />
- data modification time<br />
- inode modification time<br />
- data access time <----<br />
<br />
The last access time is a weird one, because it turns every read access into a write to the inode<br />
<br />
This wasn't a problem with traditional metal oxide mechanical hard drives, but with SSDs it is a big problem<br />
- because writes are much more expensive and wear out the device<br />
(hence wear leveling)<br />
<br />
the "noatime" option means never update the access timestamp, so it becomes invalid<br />
<br />
"relatime" means update access timestamp only occassionally (once a day) and to make sure it is always equal or newer than the modified timestamp<br />
<br />
If you have lots of small files, on some filesystems you can get to a point where you can't create new files even though there are free blocks<br />
- because you've run out of pre-allocated inodes, and the filesystem doesn't<br />
support dynamically adding them<br />
<br />
On mechanical hard drives, inode access always slowed things down because they normally caused the drive head to move between the inode location and data blocks<br />
- this is why inode data is aggressively cached, even though caching can lead to corrupted filesystem data<br />
(journaled filesystems are there to make inode caching more reliable)<br />
<br />
How does GFS deal with metadata?<br />
- centralized in the master node<br />
- trade-off is no support for small files<br />
<br />
Ceph wants to be POSIX compliant<br />
- so arbitrary support for small and large files<br />
- standard file semantics<br />
<br />
Ceph directly addresses the metadata problem in a few ways<br />
- metadata cluster to separate the load, ensure reliability<br />
- CRUSH - remove the need for clients to talk to metadata servers<br />
for every file update<br />
- slows down clients because for regular file operations they need to<br />
talk to the OSDs and metadata servers<br />
- dynamic subtree partitioning - change which metadata servers are responsible for which filesystem subtrees based on load<br />
<br />
So CRUSH is a hash function for determining where data is placed<br />
- map files to objects<br />
<br />
naive way of doing this would be like a standard hash function for a hash table<br />
- this is what is in the "uniform" method of object placement<br />
- problem with this is any changes to cluster structure could require all data to be moved<br />
- so it has other placement strategies, such as the binary tree one,<br />
where placements are much more stable<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-13&diff=24388DistOS 2023W 2023-03-132023-03-13T15:35:45Z<p>Soma: Created page with "==Discussion Questions== * List all the terms and acronyms in the Ceph paper and discus their meaning and relationship with each other. * To what degree is Ceph POSIX compliant? Is there a cost for this? * Discuss Figures 1-3 in Crush, what does each say?"</p>
<hr />
<div>==Discussion Questions==<br />
<br />
* List all the terms and acronyms in the Ceph paper and discus their meaning and relationship with each other.<br />
* To what degree is Ceph POSIX compliant? Is there a cost for this?<br />
* Discuss Figures 1-3 in Crush, what does each say?</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_16&diff=24387Mobile Apps 2023W Lecture 162023-03-10T22:16:57Z<p>Soma: Created page with "==Notes== <pre> March 10 -------- I have a hostname, want to insert it into the current HNTree - split the hostname into a bunch of components (www, carleton, ca) First, initially, we'll create a root node representing . (top of the DNS hierarchy). So that should get its own dedicated constructor. Then we'll have an add operation which will take a hostname, and it will add child hostnames add will split the hostname into its components, see if the last-most su..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
March 10<br />
--------<br />
<br />
I have a hostname, want to insert it into the current HNTree<br />
- split the hostname into a bunch of components<br />
(www, carleton, ca)<br />
<br />
First, initially, we'll create a root node representing . (top of the DNS hierarchy). So that should get its own dedicated constructor.<br />
<br />
Then we'll have an add operation which will take a hostname, and it will add child hostnames<br />
<br />
add will split the hostname into its components, see if the last-most suffix <br />
<br />
www.carleton.ca is the same as www.carleton.ca.<br />
</pre><br />
<br />
==Code==<br />
<br />
<syntaxhighlight lang="kotlin" line><br />
package carleton.comp2601.historyviewer<br />
<br />
class HNTree {<br />
var hostComponent: String<br />
var hostSuffix: String<br />
var parent: HNTree<br />
var children: MutableList<HNTree><br />
<br />
constructor() {<br />
/* Creates the "root" node by default, rest are created using<br />
different constructor<br />
*/<br />
hostComponent = ""<br />
hostSuffix = "."<br />
parent = this<br />
children = mutableListOf()<br />
}<br />
<br />
fun add(hostPrefix: String) {<br />
var allComponents: List<String><br />
var component: String<br />
var child: HNTree? = null<br />
<br />
/* still have to deal with getting to the end */<br />
<br />
allComponents = hostPrefix.split(".")<br />
component = allComponents.last()<br />
<br />
if (component == hostPrefix) {<br />
return<br />
}<br />
<br />
for (c in this.children) {<br />
if (c.hostComponent == component) {<br />
child = c<br />
break<br />
}<br />
}<br />
<br />
if (child == null) {<br />
child = HNTree()<br />
child.parent = this<br />
child.hostComponent = component<br />
child.hostSuffix = "$component.$hostSuffix"<br />
children.add(child)<br />
}<br />
<br />
child.add(allComponents.dropLast(1).joinToString())<br />
}<br />
}<br />
</syntaxhighlight></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24386Mobile Applications (Winter 2023)2023-03-10T22:15:39Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
===March 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 16|Browser History 3]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_15&diff=24385Mobile Apps 2023W Lecture 152023-03-09T04:43:19Z<p>Soma: Created page with "==Notes== <pre> March 8 ------- Last time, we were talking about how to store web browser history so we can view it in interesting ways. class HostPart { value: String, expanded: bool, higher: HostPart?, /* can be undefined */ lower: HostPart? } .ca .carleton .google www brightspace www mail sheets class HNTree { hostComponent: String, /* <-- "carleton" in "www.carleton.ca"..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
March 8<br />
-------<br />
<br />
Last time, we were talking about how to store web browser history so we can view it in interesting ways.<br />
<br />
class HostPart {<br />
value: String,<br />
expanded: bool,<br />
higher: HostPart?, /* can be undefined */<br />
lower: HostPart?<br />
}<br />
<br />
<br />
.ca<br />
<br />
.carleton .google<br />
<br />
www brightspace www mail sheets<br />
<br />
class HNTree {<br />
hostComponent: String, /* <-- "carleton" in "www.carleton.ca" */<br />
hostSuffix: String, /* <-- "carleton.ca" in "www.carleton.ca" */<br />
HostName: bool, /* is this part of a hostname, or the whole thing */<br />
hostHistory: hostHistory? /* reference to history associated with this host,<br />
only valid if HostName is true */<br />
expanded: bool, <br />
children: List<<HNTree>>,<br />
parent: HNTree,<br />
peerIndex: int /* to allow fast location of next child */<br />
historyLevel: int /* to determine how to display this relative to other history elements */<br />
<br />
fun getValue(): String {<br />
return this.value;<br />
} <br />
<br />
fun expanded?() {}<br />
<br />
/* want to get a list of host entries that are pruned to the current expanded level*/<br />
<br />
fun nextVisibleNode() {<br />
<br />
}<br />
<br />
fun nextChildNode(child): HNTree? {<br />
/* takes a child node in of this node */<br />
/* get's child's peerIndex and returns peerIndex + 1 entry in children */<br />
/* if child is not a child of this nodee, then return NULL */<br />
}<br />
<br />
fun addChild(node);<br />
/* always add at sorted position amongst children */<br />
<br />
fun delChild(index);<br />
/* have to update peerIndex of all children with index greater than this */<br />
<br />
}<br />
<br />
So we could just call listVisibleChildren on the root and we'd get exactly what we want to show. But actually we only want to generate a list of what is visible on the screen, want to use LazyColumn<br />
- also want to be able to handle *lots* of history, that's what would be useful to visualize<br />
<br />
Start:<br />
.<br />
.com<br />
.ca<br />
.net<br />
.org<br />
<br />
Tap on .ca<br />
<br />
.<br />
.com<br />
.ca<br />
.carleton.ca<br />
.net<br />
.org<br />
<br />
<br />
We need a list of items to be displayed, and we need to be able to say the first item in the list to be displayed. So we'd say VisibleHistoryItems(n, c), and it would return a list of items.<br />
<br />
At the top of the screen is always an HNTree. We can just keep calling nextVisibleNode on this element to see what should next be shown. Internally this will use nextChildNode and check whether it is expanded or not, and based on that it will return the next node to be displayed.<br />
<br />
How to test?<br />
<br />
Start off with constant data, an array of hostnames. Get it from a website or from my own browser history<br />
<br />
Then we want to parse it into our data structure, and then build a LazyColumn that displays the data structure<br />
<br />
So first<br />
- show you can just make a scrollable list of the hardcoded hostnames using LazyColumn<br />
- make a trivial version of this class that just outputs stuff from the hostnames list, but uses desired API<br />
- make the API do the right thing<br />
<br />
You want to get API right before semantics<br />
- because semantics may have to change based on needed API<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24384Mobile Applications (Winter 2023)2023-03-08T19:03:31Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
===March 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 15|Browser History 2]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_14&diff=24383Mobile Apps 2023W Lecture 142023-03-08T19:02:50Z<p>Soma: Created page with "==Notes== <pre> March 3 ------- I want to build a browser history viewer. It will allow surfing history to be viewed in a few ways: - list of pages visited (easy, dump stored history) - list "sessions" - group by what is first entered in the URL bar, whether it be a search or a direct entry of a URL - show as a grouped list (with each session collapsable) - show time of start of session, time of each subsequent request (potentially hide times)..."</p>
<hr />
<div>==Notes==<br />
<pre><br />
March 3<br />
-------<br />
<br />
I want to build a browser history viewer. It will allow surfing history to be viewed in a few ways:<br />
- list of pages visited (easy, dump stored history)<br />
- list "sessions"<br />
- group by what is first entered in the URL bar, whether it be a<br />
search or a direct entry of a URL<br />
- show as a grouped list (with each session collapsable)<br />
- show time of start of session, time of each subsequent request<br />
(potentially hide times)<br />
- What "places" I visit<br />
- can group by hostname<br />
- but sometimes hostname is too specific<br />
(subdomains can get added in weird ways)<br />
- maybe make it multilevel?<br />
<br />
For example, consider the place "carleton.ca"<br />
- I visit many other places inside of Carleton<br />
- some are in SCS: openstack.scs.carleton.ca, homeostasis.scs.carleton.ca<br />
- so maybe make a hierarchical grouping of hostnames?<br />
<br />
<br />
Collapsed view:<br />
- google.com +<br />
- carleton.ca +<br />
<br />
Expanded view:<br />
- google.com<br />
- mail.google.com<br />
- sheets.google.com<br />
- carleton.ca<br />
- scs.carleton.ca<br />
- brightspace.carleton.ca<br />
<br />
So we want a data structure that extracts the tree inherent in hostnames<br />
<br />
Do you all know how DNS works?<br />
<br />
Remember that DNS is a distributed hierarchical fault-tolerant database for mapping hostnames to IP addresses and other info<br />
<br />
While there is lots of caching, basic lookup works like this:<br />
<br />
* look up root server (.)<br />
* ask root server for first component (end of URL)<br />
(.com)<br />
* go to that server, ask for next component<br />
(ask .com server for google)<br />
* continue recursively until you fully resolve host (e.g., www.google.com)<br />
<br />
<br />
<br />
So we need to split up the hostname into its components, and build a tree of those components.<br />
<br />
Linear time tree generation<br />
- you do this in 2402<br />
<br />
But...we want to display this, so we don't want a data structure that is just for us. We need to understand what data structure our display widget wants!<br />
<br />
We can just make a simple lazy scrollable list that we can interact with to expand as needed<br />
<br />
So we have a tree data structure, and for each node we have a property which says whether it is expanded or not<br />
- if it is expanded, we display the children. If not, we don't.<br />
<br />
<br />
.ca<br />
<br />
.carleton .google<br />
<br />
www brightspace www mail sheets<br />
<br />
So our initial view would just show .ca, or it would show carleton.ca and google.ca<br />
<br />
If we tap any of these, the node gets marked to expanded. Then the list will automatically expand to show the nodes under that current one.<br />
<br />
<br />
So how should we make a tree in Kotlin?<br />
<br />
We'll have to specify the class of the node elements<br />
<br />
class HostPart {<br />
value: String,<br />
expanded: bool,<br />
higher: HostPart?, /* can be undefined */<br />
lower: HostPart?<br />
}<br />
<br />
Each level of tree is a List<<HostPart>><br />
<br />
Need to figure out how to react to tap events so we can expand and shrink our displayed list<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24382Mobile Applications (Winter 2023)2023-03-08T19:02:35Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
===March 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 14|Browser History 1]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-08&diff=24381DistOS 2023W 2023-03-082023-03-08T18:17:51Z<p>Soma: Created page with "No in-class notes, see class video on Brightspace/Zoom."</p>
<hr />
<div>No in-class notes, see class video on Brightspace/Zoom.</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-06&diff=24378DistOS 2023W 2023-03-062023-03-06T18:18:37Z<p>Soma: /* Notes */</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
NASD & Tapestry<br />
---------------<br />
<br />
What problem(s) is NASD designed to solve?<br />
<br />
What's the standard architecture for a file server?<br />
- you have a server with attached storage<br />
- that server reads storage then sends what it reads over the network to a client<br />
<br />
Limitations of this approach<br />
- bandwidth bottleneck on the server: reading lots of disks in parallel only to copy to memory then send over the network puts huge strain on server memory system<br />
- also limits clients inherently, again because of bandwidth constraints<br />
<br />
Idea: have clients talk directly to the disks, server just has to authorize access<br />
<br />
Why aren't the disks just smaller servers? Because they don't understand the concept of files<br />
- traditionally this abstraction is blocks, but NASD does something different, why?<br />
<br />
<br />
What problem is Tapestry designed to solve?<br />
- routing when you don't want to retrieve information from fixed hosts<br />
- need to propagate messages to find who has what information<br />
- also, hosts will come and go, so need to deal with that<br />
<br />
WHEN is this problem relevant? When is it not?<br />
<br />
How well does this kind of architecture deal with untrusted or even malicious nodes?<br />
<br />
Big question: do you think we use these kinds of architectures so much nowadays? Why or why not?<br />
<br />
<br />
<br />
AFTER GROUP DISCUSSIONS<br />
<br />
How does regular network routing work (at a high level)?<br />
<br />
First, you have LANs<br />
- broadcast domains for figuring out mapping of IP<->MAC address, ARP<br />
- send out a message saying "who has X.X.X.X?" and someone replies "I do, you can reach me at this address (specific to WiFi, Ethernet, etc)<br />
- between broadcast domains, we have routing tables<br />
- some hosts are routers, meaning that they know how to reach<br />
hosts beyond the LAN<br />
- routers have spanning trees that map IP addresses to destinations<br />
- for a home router there isn't much to it, it just forwards all packets not destined for the LAN to another gateway<br />
- but once you get into the core of the Internet, routers have multiple interfaces and muliple ways to forward packets, and routing tables are the instructions for how to do this forwarding<br />
- protocols like BGP are for updating routing tables<br />
<br />
<br />
This is all based on IP addresses, and those are uniquely associated with a host (except for NAT)<br />
<br />
When we talk about overlay networks, we want to route based on criteria other than IP address (i.e., beyond host-based identification)<br />
<br />
Often we want to do content-based routing, i.e., exchange information without caring who has that information, let the network figure it out<br />
- kind of like how when we communicate with an IP address we don't think about the path data has to flow to get to that IP address<br />
<br />
Why are overlay networks, DHT's important for distributed operating systems?<br />
- we often want to route information correctly even when hosts aren't working<br />
- with an overlay network, hosts can have responsibilities that can be shifted if they don't respond, or responsibility can be distributed so failures don't matter (3 hosts have the info, if two are down one can still respond)<br />
<br />
back to NASD, I said the point of the architecture is dealing with bandwidth constraints, but can't we solve that another way?<br />
- seems really complicated right?<br />
<br />
<br />
Classic file server:<br />
- client requests data from server<br />
- server returns data from client<br />
<br />
but with NASD-type systems<br />
- client requests info on data from metadata server (low bandwidth)<br />
- server tells client where to find the data<br />
- client communicates with data servers to get data (high bandwidth)<br />
<br />
Why can't we just have lots of metadata servers too? Why have fewer?<br />
- because metadata is where concurrency issues happen a lot<br />
- so "scaling up" metadata access is much harder<br />
- and besides, it often isn't needed<br />
<br />
Where does security come into this?<br />
- note that security is normally enforced at the metadata level<br />
(i.e., when you open a file)<br />
- but with a NASD-type architecture, data servers have to do security in another way (e.g., capabilities) or they don't do security at all<br />
<br />
Most of the systems we'll discuss this term assume servers are trusted<br />
- and sometimes even assume clients<br />
- because they are for solutions within organizations where they have other<br />
security mechanisms protecting them from the "outside"<br />
- thus they implement very little security within them<br />
<br />
Traditionally we don't enforce security within the hardware of a system<br />
- communication between storage devices and the CPU happen "in the clear"<br />
- same idea, applied in a distributed system<br />
- the network really is the computer<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-06&diff=24377DistOS 2023W 2023-03-062023-03-06T16:48:33Z<p>Soma: Created page with "==Notes== <pre> NASD & Tapestry --------------- What problem(s) is NASD designed to solve? What's the standard architecture for a file server? - you have a server with attached storage - that server reads storage then sends what it reads over the network to a client Limitations of this approach - bandwidth bottleneck on the server: reading lots of disks in parallel only to copy to memory then send over the network puts huge strain on server memory system - also l..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
NASD & Tapestry<br />
---------------<br />
<br />
What problem(s) is NASD designed to solve?<br />
<br />
What's the standard architecture for a file server?<br />
- you have a server with attached storage<br />
- that server reads storage then sends what it reads over the network to a client<br />
<br />
Limitations of this approach<br />
- bandwidth bottleneck on the server: reading lots of disks in parallel only to copy to memory then send over the network puts huge strain on server memory system<br />
- also limits clients inherently, again because of bandwidth constraints<br />
<br />
Idea: have clients talk directly to the disks, server just has to authorize access<br />
<br />
Why aren't the disks just smaller servers? Because they don't understand the concept of files<br />
- traditionally this abstraction is blocks, but NASD does something different, why?<br />
<br />
<br />
What problem is Tapestry designed to solve?<br />
- routing when you don't want to retrieve information from fixed hosts<br />
- need to propagate messages to find who has what information<br />
- also, hosts will come and go, so need to deal with that<br />
<br />
WHEN is this problem relevant? When is it not?<br />
<br />
How well does this kind of architecture deal with untrusted or even malicious nodes?<br />
<br />
Big question: do you think we use these kinds of architectures so much nowadays? Why or why not?<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Distributed_OS:_Winter_2023&diff=24376Distributed OS: Winter 20232023-03-06T16:31:29Z<p>Soma: /* Class Schedule & Readings */</p>
<hr />
<div>==Course Outline==<br />
<br />
[[Distributed OS: Winter 2023 Course Outline|Here]] is the course outline.<br />
<br />
==Project Help==<br />
<br />
To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.<br />
<br />
To begin selecting a paper, I suggest that you:<br />
* search on Google Scholar using keywords relating to your interests, and/or<br />
* browse the proceedings of major conferences that publish work related to distributed operating systems.<br />
<br />
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].<br />
<br />
To help you write a literature review or the background of a research paper, read the following:<br />
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]<br />
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]<br />
<br />
==Class Schedule & Readings==<br />
<br />
===[[DistOS 2023W 2023-01-09|January 9, 2023]]===<br />
<br />
Introduction<br />
<br />
===[[DistOS 2023W 2023-01-11|January 11, 2023]]===<br />
<br />
Designing a distributed operating system<br />
<br />
===[[DistOS 2023W 2023-01-16|January 16, 2023]]===<br />
<br />
The Early Internet & Multics:<br />
* [https://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]<br />
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video<br />
Optional: Browse around [https://www.multicians.org/ the Multicians website].<br />
<br />
===[[DistOS 2023W 2023-01-18|January 18, 2023]]===<br />
<br />
UNIX<br />
* [https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/ Richard Jensen, "Unix at 50: How the OS that powered smartphones started from failure" (2019, Ars Technica)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]<br />
* [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video<br />
Note that the video covers the main points of the UNIX paper.<br />
<br />
===[[DistOS 2023W 2023-01-23|January 23, 2023]]===<br />
<br />
The Mother of All Demos:<br />
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].<br />
* [https://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]<br />
<br />
The Alto:<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])<br />
<br />
===[[DistOS 2023W 2023-01-25|January 25, 2023]]===<br />
<br />
Distributed Shared Memory<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/kai-li-shared.pdf Kai Li, "Shared virtual memory on loosely coupled multiprocessors" (1986)] '''(OPTIONAL)'''<br />
<br />
===[[DistOS 2023W 2023-01-30|January 30, 2023]]===<br />
<br />
Remote Procedure Calls<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/nelson1981-rpc.pdf Bruce J. Nelson, "Remote Procedure Call" (1981)] (Read the summary starting on p. 224, look at the Intro and Background if you have time.)<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/rfc1050.txt Sun Microsystems, "RPC: Remote Procedure Call Protocol Specification" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-01|February 1, 2023]]===<br />
<br />
LOCUS & NFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]<br />
<br />
===[[DistOS 2023W 2023-02-06|February 6, 2023]]===<br />
<br />
Sprite, AFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-08|February 8, 2023]]===<br />
<br />
Plan 9 & Inferno<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2018f/doward97-inferno.pdf Doward et al., The Inferno Operating System (1997)]<br />
* [http://www.inferno-os.info/inferno/ Inferno OS website] (browse)<br />
* [http://doc.cat-v.org/inferno/4th_edition/styx Rob Pike and Dennis M. Ritchie, "The Styx Architecture for Distributed Systems"]<br />
<br />
===[[DistOS 2023W 2023-02-13|February 13, 2023]]===<br />
<br />
Midterm review<br />
<br />
===February 15, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/~soma/distos/2023w/comp4000-2023w-midterm.pdf Midterm exam]<br />
<br />
===[[DistOS 2023W 2023-02-27|February 27, 2023]]===<br />
<br />
"Web Scale" discussion<br />
<br />
===[[DistOS 2023W 2023-03-01|March 1, 2023]]===<br />
<br />
OceanStore & BOINC<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]<br />
<br />
Project Proposal due March 3rd<br />
<br />
===[[DistOS 2023W 2023-03-06|March 6, 2023]]===<br />
<br />
NASD & Tapestry<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]<br />
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]<br />
<br />
Background (optional but helpful):<br />
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]<br />
<br />
===[[DistOS 2023W 2023-03-08|March 8, 2023]]===<br />
<br />
Chubby & GFS<br />
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]<br />
* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]<br />
<br />
===[[DistOS 2023W 2023-03-13|March 13, 2023]]===<br />
<br />
Ceph<br />
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]<br />
<br />
===[[DistOS 2023W 2023-03-15|March 15, 2023]]===<br />
<br />
BigTable & MapReduce<br />
* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]<br />
* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)<br />
<br />
===[[DistOS 2023W 2023-03-20|March 20, 2023]]===<br />
<br />
Omega & Borg<br />
* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]<br />
* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]<br />
<br />
===[[DistOS 2023W 2023-03-22|March 22, 2023]]===<br />
<br />
Cassandra & Dynamo<br />
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]<br />
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]<br />
<br />
===[[DistOS 2023W 2023-03-27|March 27, 2023]]===<br />
<br />
Zookeeper & Delos<br />
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]<br />
* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]<br />
<br />
===[[DistOS 2023W 2023-03-29|March 29, 2023]]===<br />
<br />
Haystack & F4<br />
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]<br />
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]<br />
<br />
===[[DistOS 2023W 2023-04-03|April 3, 2023]]===<br />
<br />
Spanner & Tensorflow<br />
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]<br />
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]<br />
<br />
===[[DistOS 2023W 2023-04-05|April 5, 2023]]===<br />
<br />
TBD<br />
<br />
===April 10, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-01&diff=24375DistOS 2023W 2023-03-012023-03-02T04:53:20Z<p>Soma: </p>
<hr />
<div>==Discussion Questions==<br />
<br />
You have until 12:10 to discuss the following:<br />
<br />
OceanStore<br />
* What is the threat model underlying OceanStore security?<br />
* How does this compare to the threat model of modern cloud storage providers?<br />
* Would you use OceanStore? Why or why not?<br />
<br />
BOINC<br />
* What was the original inspiration for this work?<br />
* What kind of problems is this style of computing suitable for? What problems is it not suitable for?<br />
* What is the threat model for BOINC-type systems? Does this threat model make it suitable or unsuitable for some applications?<br />
* Why aren't BOINC-style systems so popular anymore?<br />
<br />
==Notes==<br />
<br />
<pre><br />
BOINC & OceanStore<br />
------------------<br />
<br />
BOINC<br />
- what is it good for? (what problems?)<br />
- embarassingly parallel problems<br />
- "work units" can be checked and combined relatively cheaply<br />
<br />
Note that with BOINC bandwidth is at a premium<br />
- want to send relatively small chunks of data to clients that require a significant amount of analysis<br />
<br />
A classic supercomputer is very good at low-latency, high bandwidth communication. BOINC isn't!<br />
<br />
BOINC is an example of "volunteer computing", people allowing their computers to be used for tasks for which they get no direct benefit<br />
- what is the malicious form of this? a botnet, mining cryptocurrency<br />
<br />
Why did volunteer computing become popular?<br />
- dream of making use of vast computing resources of computers attached to the Internet that are otherwise not doing much else<br />
- note most were desktop systems that were left on for long periods of time running screensavers<br />
- "academic nature" of the past Internet<br />
<br />
But nowadays we care about power, we worry about malware, and researchers aren't demanding access to these resources (they can have their own computers that can do their own data analysis much more easily)<br />
<br />
But even with BOINC, we have a trust issue<br />
- clients may not trust projects, but that isn't an issue in practice that much<br />
- but projects cannot trust clients...because of gamification<br />
- some will want to get higher scores so they can get to the top of leaderboards<br />
- so project must be able to verify whether clients did the work they say<br />
they've done<br />
- general solution: send work units to mulitple computers, make sure they agree with their answers<br />
- don't have to do it for all work units, just enough to detect cheaters<br />
<br />
<br />
BOINC makes use of lots of untrusted computers to do large-scale computation<br />
OceanStore makes use of lots of untrusted computers to do large-scale storage<br />
<br />
Trust model for OceanStore is that storage computers aren't trusted<br />
- data is replicated and encrypted, spread across mulitple systems<br />
<br />
In some sense OceanStore is very familiar<br />
- we store data on giant sets of remote systems<br />
- but the trust model is VERY different<br />
- the party you pay is the one that controls all of the storage<br />
<br />
<br />
If OceanStore was proposed today, storage would be paid for with cyptocurrency probably. But it had no monitization model, assumed people would just make contracts to figure it out. (To be fair it was a research project not a commercial endeavor.)<br />
<br />
It turns out that in business, it is mostly best to just have one entity handle the payment and the work. So that's why we have giant cloud providers and not distributed computing and storage.<br />
<br />
If your infrastructure is trusted (i.e., the one you pay controls the resources and can give you guarantees), you don't need all the overhead of dealing with untrusted systems.<br />
<br />
Cryptocurrency, blockchain-based systems are trying to bring back some of this decentralization, but at huge efficiency costs and many other problems.<br />
- the trust issue never goes away in distributed systems<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Apps_2023W_Lecture_13&diff=24374Mobile Apps 2023W Lecture 132023-03-02T04:52:38Z<p>Soma: Created page with "==Notes== <pre> Android Architecture & Security ------------------------------- * I've graded all proposals * On Friday, progress report 3 is due * If you have something to demo, please demo in Friday! What is running on an Android device? How does it compare to a regular Linux system? Android documentation uses lots of Android-specific jargon, not at all clear how it maps to more normal concepts. So I'm going to try explaining that here. First, Android runs on L..."</p>
<hr />
<div>==Notes==<br />
<br />
<pre><br />
Android Architecture & Security<br />
-------------------------------<br />
<br />
* I've graded all proposals<br />
* On Friday, progress report 3 is due<br />
* If you have something to demo, please demo in Friday!<br />
<br />
<br />
What is running on an Android device? How does it compare to a regular Linux system?<br />
<br />
Android documentation uses lots of Android-specific jargon, not at all clear how it maps to more normal concepts. So I'm going to try explaining that here.<br />
<br />
First, Android runs on Linux. So what does that mean?<br />
<br />
Standard division for most Linux systems:<br />
<br />
Applications<br />
------------<br />
Linux Kernel<br />
------------<br />
Hardware<br />
<br />
<br />
Kernel communicates with hardware using hardware APIs (accessing special memory locations, I/O ports, interrupts, etc)<br />
<br />
Applications talk to the Linux kernel using system calls.<br />
<br />
Applications run inside of processes. A process is an address space + one or more execution context. Each address space is separate, processes cannot access the memory of other processes (unless explicitly shared).<br />
<br />
When you run "ps" or "top" on a Linux system, you can see all of the processes that are running.<br />
<br />
You can do the same on Android as well, underneath it is just Linux.<br />
But...Android has a bunch of extra stuff.<br />
<br />
<br />
One key difference is in how processes talk to each other. Standard Linux supports multiple communication mechanisms:<br />
- TCP/IP sockets<br />
- System V IPC<br />
- pipes<br />
- shared memory (using mmap)<br />
<br />
Android adds its own custom IPC mechanism, Binder. Yes, they *patched* Linux to add support for a new IPC mechanism!<br />
- this is the mechanism that facilitates communication in Android<br />
- most Android APIs are built on this when they go between processes<br />
- key feature: it is very fast! (uses shared memory, but with kernel support)<br />
<br />
Originally Binder wasn't in the main Linux kernel, it was kept "out of tree", but it has been upstreamed and so modern Linux systems do have support for Binder.<br />
<br />
<br />
On a standard Linux system, permissions are all based on files and users. Devices are special files, and access is granted or not based on the user and group of a process.<br />
<br />
User/group file permissions are not flexible enough for a system like Android, they wanted something else to determine what applications could access what.<br />
<br />
For example, how to control access to a camera? The raw camera would show up as a device in /dev, how can we make sure only one app is using the camera at a time, that permission is only granted when the user says so and the user can withdraw it?<br />
<br />
Well, every app on Android has its own user ID, and that is used to limit the access the app has.<br />
- not per user, per APP!<br />
<br />
This user-ID-per-app thing is part of the Android *application sandbox* - applications have only limited access to files and other resources. They are confined by default.<br />
<br />
Web browsers also implement a sandbox, that's why each tab cannot access data in another tab (for JavaScript code), but it is implemented using very different technologies and with different levels of access<br />
- remember a sandbox is an aspiration, not a specific set of technologies!<br />
<br />
If a regular Android app asks the Linux kernel for access to the camera, the kernel will just say no because it won't have the right permissions (it won't have the right user and group ID for access). But then how does it get access?<br />
<br />
Android has special background processes, "servers", that wait for requests from Android applications and then do the work on behalf of the app.<br />
- these services perform a role somewhat like that of the Linux kernel,<br />
but in an Android-specific way<br />
- most central one is system_server, this implements the basic plumbing,<br />
but there are others like media_server<br />
<br />
Note that the camera device's permissions are setup so that an android server process has exclusive access. Applications then ask it (via intents which are Binder-relayed messages) for access to these resources and the server can say yes or no. Because Binder is fast this interaction is also fast.<br />
<br />
<br />
What's cool about Binder is you don't have to know which process you want to talk to. You can broadcast a request with specific criteria and the appropriate processes (applications or servers) will get it and respond as they see fit.<br />
- this is all orchestrated by system_server with the help of the Linux kernel's binder mechanism<br />
<br />
Why does all this matter?<br />
- to some degree it doesn't, if you're doing normal application development<br />
- but when things break, it is important to know who is telling you that there is a problem and where you can potentially observe things to try and fix them<br />
</pre></div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Mobile_Applications_(Winter_2023)&diff=24373Mobile Applications (Winter 2023)2023-03-02T04:52:13Z<p>Soma: /* Class Schedule */</p>
<hr />
<div>==Course Information==<br />
<br />
* [[Mobile Applications (Winter 2023) Course Outline|Course outline]]<br />
* [[Mobile Applications (Winter 2023) Project Guidelines|Project Guidelines]]<br />
<br />
==Class Schedule==<br />
<br />
===January 11, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 1|Introduction]]<br />
<br />
===January 13, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 2|Jetpack Compose]]<br />
<br />
===January 18, 2023===<br />
<br />
Intro to Tutorial 1<br />
<br />
===January 20, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 4|Images, Gestures]]<br />
<br />
===January 27, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/wiki/index.php/Mobile_Apps_2023W_Lecture_6 Embedding Views & WebViews in Compose]<br />
<br />
Progress Report 1 due<br />
<br />
===February 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 7|Processes & Threads]]<br />
<br />
===February 3, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 8|REST APIs]]<br />
<br />
===February 8, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 9|Project Planning, SQLite]]<br />
<br />
===February 10, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 10|Designing the database]]<br />
<br />
===February 15, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 11|Working with databases]]<br />
<br />
===February 17, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 12|Scrolling and databases]]<br />
<br />
Project Proposal due<br />
<br />
===March 1, 2023===<br />
<br />
[[Mobile Apps 2023W Lecture 13|Android Architecture and Security]]<br />
<br />
==Tutorials==<br />
<br />
Tutorials should normally be checked off within a week of them being presented in class.<br />
<br />
* [[Mobile Apps 2023W Tutorial 1|Tutorial 1]] (January 18, 2023)<br />
* [[Mobile Apps 2023W Tutorial 2|Tutorial 2]] (January 25, 2023)</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-01&diff=24372DistOS 2023W 2023-03-012023-03-01T16:39:41Z<p>Soma: /* Discussion Questions */</p>
<hr />
<div>==Discussion Questions==<br />
<br />
You have until 12:10 to discuss the following:<br />
<br />
OceanStore<br />
* What is the threat model underlying OceanStore security?<br />
* How does this compare to the threat model of modern cloud storage providers?<br />
* Would you use OceanStore? Why or why not?<br />
<br />
BOINC<br />
* What was the original inspiration for this work?<br />
* What kind of problems is this style of computing suitable for? What problems is it not suitable for?<br />
* What is the threat model for BOINC-type systems? Does this threat model make it suitable or unsuitable for some applications?<br />
* Why aren't BOINC-style systems so popular anymore?</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2023W_2023-03-01&diff=24371DistOS 2023W 2023-03-012023-03-01T16:38:09Z<p>Soma: Created page with "==Discussion Questions== OceanStore * What is the threat model underlying OceanStore security? * How does this compare to the threat model of modern cloud storage providers? * Would you use OceanStore? Why or why not? BOINC * What was the original inspiration for this work? * What kind of problems is this style of computing suitable for? What problems is it not suitable for? * What is the threat model for BOINC-type systems? Does this threat model make it suitable o..."</p>
<hr />
<div>==Discussion Questions==<br />
<br />
OceanStore<br />
* What is the threat model underlying OceanStore security?<br />
* How does this compare to the threat model of modern cloud storage providers?<br />
* Would you use OceanStore? Why or why not?<br />
<br />
BOINC<br />
* What was the original inspiration for this work?<br />
* What kind of problems is this style of computing suitable for? What problems is it not suitable for?<br />
* What is the threat model for BOINC-type systems? Does this threat model make it suitable or unsuitable for some applications?<br />
* Why aren't BOINC-style systems so popular anymore?</div>Somahttps://homeostasis.scs.carleton.ca/wiki/index.php?title=Distributed_OS:_Winter_2023&diff=24370Distributed OS: Winter 20232023-03-01T16:35:34Z<p>Soma: /* Class Schedule & Readings */</p>
<hr />
<div>==Course Outline==<br />
<br />
[[Distributed OS: Winter 2023 Course Outline|Here]] is the course outline.<br />
<br />
==Project Help==<br />
<br />
To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.<br />
<br />
To begin selecting a paper, I suggest that you:<br />
* search on Google Scholar using keywords relating to your interests, and/or<br />
* browse the proceedings of major conferences that publish work related to distributed operating systems.<br />
<br />
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].<br />
<br />
To help you write a literature review or the background of a research paper, read the following:<br />
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]<br />
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]<br />
<br />
==Class Schedule & Readings==<br />
<br />
===[[DistOS 2023W 2023-01-09|January 9, 2023]]===<br />
<br />
Introduction<br />
<br />
===[[DistOS 2023W 2023-01-11|January 11, 2023]]===<br />
<br />
Designing a distributed operating system<br />
<br />
===[[DistOS 2023W 2023-01-16|January 16, 2023]]===<br />
<br />
The Early Internet & Multics:<br />
* [https://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]<br />
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video<br />
Optional: Browse around [https://www.multicians.org/ the Multicians website].<br />
<br />
===[[DistOS 2023W 2023-01-18|January 18, 2023]]===<br />
<br />
UNIX<br />
* [https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/ Richard Jensen, "Unix at 50: How the OS that powered smartphones started from failure" (2019, Ars Technica)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]<br />
* [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video<br />
Note that the video covers the main points of the UNIX paper.<br />
<br />
===[[DistOS 2023W 2023-01-23|January 23, 2023]]===<br />
<br />
The Mother of All Demos:<br />
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].<br />
* [https://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]<br />
<br />
The Alto:<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])<br />
<br />
===[[DistOS 2023W 2023-01-25|January 25, 2023]]===<br />
<br />
Distributed Shared Memory<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/kai-li-shared.pdf Kai Li, "Shared virtual memory on loosely coupled multiprocessors" (1986)] '''(OPTIONAL)'''<br />
<br />
===[[DistOS 2023W 2023-01-30|January 30, 2023]]===<br />
<br />
Remote Procedure Calls<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/nelson1981-rpc.pdf Bruce J. Nelson, "Remote Procedure Call" (1981)] (Read the summary starting on p. 224, look at the Intro and Background if you have time.)<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/rfc1050.txt Sun Microsystems, "RPC: Remote Procedure Call Protocol Specification" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-01|February 1, 2023]]===<br />
<br />
LOCUS & NFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]<br />
<br />
===[[DistOS 2023W 2023-02-06|February 6, 2023]]===<br />
<br />
Sprite, AFS<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]<br />
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]<br />
<br />
===[[DistOS 2023W 2023-02-08|February 8, 2023]]===<br />
<br />
Plan 9 & Inferno<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2018f/doward97-inferno.pdf Doward et al., The Inferno Operating System (1997)]<br />
* [http://www.inferno-os.info/inferno/ Inferno OS website] (browse)<br />
* [http://doc.cat-v.org/inferno/4th_edition/styx Rob Pike and Dennis M. Ritchie, "The Styx Architecture for Distributed Systems"]<br />
<br />
===[[DistOS 2023W 2023-02-13|February 13, 2023]]===<br />
<br />
Midterm review<br />
<br />
===February 15, 2023===<br />
<br />
[https://homeostasis.scs.carleton.ca/~soma/distos/2023w/comp4000-2023w-midterm.pdf Midterm exam]<br />
<br />
===[[DistOS 2023W 2023-02-27|February 27, 2023]]===<br />
<br />
"Web Scale" discussion<br />
<br />
===[[DistOS 2023W 2023-03-01|March 1, 2023]]===<br />
<br />
OceanStore & BOINC<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]<br />
<br />
Project Proposal due March 3rd<br />
<br />
===March 6, 2023===<br />
<br />
NASD & Tapestry<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]<br />
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]<br />
<br />
Background (optional but helpful):<br />
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]<br />
<br />
===March 8, 2023===<br />
<br />
Chubby & GFS<br />
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]<br />
* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]<br />
<br />
===March 13, 2023===<br />
<br />
Ceph<br />
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].<br />
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]<br />
<br />
===March 15, 2023===<br />
<br />
BigTable & MapReduce<br />
* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]<br />
* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)<br />
<br />
===March 20, 2023===<br />
<br />
Omega & Borg<br />
* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]<br />
* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]<br />
<br />
===March 22, 2023===<br />
<br />
Cassandra & Dynamo<br />
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]<br />
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]<br />
<br />
===March 27, 2023===<br />
<br />
Zookeeper & Delos<br />
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]<br />
* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]<br />
<br />
===March 29, 2023===<br />
<br />
Haystack & F4<br />
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]<br />
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]<br />
<br />
===April 3, 2023===<br />
<br />
Spanner & Tensorflow<br />
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]<br />
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]<br />
<br />
===April 5, 2023===<br />
<br />
TBD<br />
<br />
===April 10, 2023===<br />
<br />
Project Presentations<br />
<br />
===April 12, 2023===<br />
<br />
Project Presentations</div>Soma