Soma-notes - User contributions [en]

Distributed OS: Fall 2017

2017-09-26T13:49:04Z

Zero: Added project reminder.

==Course Outline==

[[Distributed OS: Fall 2017 Course Outline|Here]] is the course outline. It should see only minor modifications during the semester.

==Assigned Readings==

===September 12, 2017===

The Early Internet:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video

The Mother of All Demos:
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].
* [http://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]

===September 14, 2017===

The Alto:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])

===September 19, 2017===

* [http://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]

Optional: Browse around [http://www.multicians.org/ the Multicians website].

===September 21, 2017===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]

===September 26, 2017===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System." (1988)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/tanenbaum-amoeba.pdf Andrew Tannenbaum et al., "The Amoeba System" (1990)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/clouds-dasgupta.pdf Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)]
* Reminder for grad students (and optionally undergrad) to start writing an abstract, outline, and find roughly 10 references for their project

===September 28, 2017===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]

==Project Help==

To develop your literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:
* search on Google Scholar using keywords relating to your interests, and/or
* browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].

To help you write a literature review or the background of a research paper, read the following:
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]

WebFund 2016W Lecture 13

2016-04-11T23:17:42Z

Zero: Added link to JSFiddle.

==Video==

The video from the lecture given on March 1, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec13-01Mar2016.mp4 is now available].

==Notes==

===In lecture===
<pre>
Lecture 13
----------

To get help with assignment 4, look at Assignment 5 from Winter 2015.

closures

DOM
- document object model
- set of JavaScript objects for accessing the current page in the browser

client-side JS can access the DOM
(data structures representing the current web page)

server-side JS can access the operating system

client-side JS is *limited* in its access
- because such access is dangerous
- ANYONE can run JavaScript in your browser
- so, JavaScript in the browser is sandboxed

execution sandbox is an environment for running untrusted programs
- sandbox limits access and resources

Client-side JavaScript has no native way to
(well, it used to...)
- access local files
- open network connections
- run multiple threads
- access other windows/programs

Originally JavaScript was so limited it could do no background processing
- could only run when the page was loaded or when the user acted

Microsoft messed it up
- Outlook Web Access
- they built an ActiveX control which implemented XMLHttpRequest()
- but really, it was a way to do GETs and POSTs in the background
- other browsers implemented the API natively

Now you could update data in a web page without reloading the page
</pre>

===Student Notes===

====storelogline.js and storelogs.js====

*To use storelogline.js:
**Find an entry from an existing log file and copy the line
**Run storelogline.js with the copied line pasted after the script name as command line parameters
**Optionally, verify that the line was entered in the database by typing the following with the Mongo client:
<code><pre>
mongo
use log-demo
db.logs.find()
</pre></code>

=====Writing storelogs.js=====

*We can use the code from storeloglines.js to start building the storelogs.js script for tutorial 5
*To read in a file specified in the command line, you do <code>var data = fs.readFileSync(process.argv[2], ‘utf-8’);</code>
**You use <code>process.argv[2]</code> because in the terminal your third argument is going to be the file to read
*To split the data into lines, yo do <code>var lines = data.split(‘\n’);</code>
*To parse the lines and store them into an array, you can do something like:

<code><pre>
var entries = [];
int i; entry, fields, j;

for(i=0;i<lines.length; i++) {
if(lines[i] &&lines[i]!==null) {
field=lines[i].split(‘ ‘); //this is used to split the lines at index i wherever the spaces occur
console.log(split); //when this is printed you’ll see that it splits between all the spaces as well—>you can regular expression to fix this but we fixed it through the while loop
entry={};
j=0;

//Used to delete the spaces
while(j<field.length) {
if(field.lenght[j]===“ “) {
field.splice(j, 1);
}else{
j++;
}
}

console.log(field); //—>this prints the logs out without the spaces
entry.date=field[0] + “ “ + field[1]; //Than you start populating the array
entry.time=field[2];
entry.host=field[3];
entry.service=field[4].slice(0,-1); //the program will have an error here without the if statement above
entry.message=filed.slice(5).join(‘ ‘);
//to store in to the entries ( the array )
entries.push(entry);
}
}
</pre></code>

*To insert into the database you don’t need a loop to iterate. You can use the <code>collection.insert()</code> method which can take an array (entries) as the argument to insert
**API’s have documentation and they change because there’s also <code>insertOne()</code> and <code>insertMany()</code> that seem to do the same thing
**For what we want to do <code>insert()</code> will work
*For querying the database (assignment 4), there is a trick
**There are previous assignment solutions that can help you.
**It's the previous assignment from last year’s assignment 5
*Note that when using special characters like [] and () in regular expression, you will need to use a backslash to escape them

====JavaScript Closures====

*JavaScript closures are about nesting functions
**For example:
<code><pre>
var f = function(x) {
function g(y) {
return x+y;
}
return g; //The function g can see x
}

var q = f(7);
var h = f(12);
console.log(“q = “ +q(4)); //prints 11
console.log(“h(4) = “ + h(4)); //Prints 16
</pre></code>

*This gives different answers because <code>x</code> is different in two two instances of <code>g()</code>
*You cannot do things like this with C
*In examforms we have an app.get for list and we have a function in it that has a callback called <code>findCallback()</code>
**This callback function has access to the request and response objects because they were in scope at the location where the callback function was defined
**To have <code>findCallback</code> have access to request and response you just need to create different instances of <code>findCallback</code> that have their own copies of request and response, therefore one version of <code>findCallback</code> will have one request and on another call it will have another request
*Closure is a concept you need to understand because it’s very handy

====Client-Side JavaScript====

*In JavaScript we’ve been doing everything on the server side so far
*Using the browser dev tools, you can go to console and play around in it. Its almost like the Node environment in a terminal but with some missing features
*In the browser there is no <code>require()</code>
*In the browser the unit is the page but in the server its the file
*You can access objects representing the page by typing <code>document</code> or <code>window</code>
*When you load a webpage, it can load and run scripts
**By running JavaScript within the browser, you can write code that has access to the webpage (in particular, the DOM)
***The DOM (Document Object Model) is a tree-like structure of objects representing the HTML elements of the page

=====Differences Between Client-Side and Server-Side JavaScript=====

*Client-side JavaScript can access the DOM (data structure repressing the current web page)
*Server-side JavaScript can access the operating system
*In client-side JavaScript, you can’t access everything... It's *limited*
**That’s because such access is dangerous, the code could be coming from ANYONE
**ANYONE can run JavaScript in your browser and you don’t want this code messing with your system
**The browser is designed to protect your system from bad JavaScript, so JavaScript in the browser is sandboxed
***An execution sandbox is an environment for running untrusted code
***The sandbox limits access and resources
***Client-side JavaScript has no native way to:
****Access local files
****Open network connections
****Run multiple threads
****Access other windows/programs
*The model of client-side JavaScript was that it should be limited
*Originally JavaScript was so limited that it could do no background processing
**It originally could only run when the webpage was loaded or when the user acted
**It was singly threaded so you had to limit your execution
**Microsoft messed it up (in particular, the Outlook Web Access team messed it up)
***They built an ActiveX control which implemented XMLHttpRequest() —> an object that would send a GET or a POST request and would bring back a document that is assumed to be XML
***But more importantly, it was a way to do GETs and POSTs in the background
***Other browsers implemented the API natively
***Now you could update data in a web page without reloading the page
***Google really ran with this (with gmail, google maps, etc)
*Client-side JavaScript is the same JavaScript we have been working with except it has access to a webpage now but you access the things you can access the exact same way
*If you want to change how the current page looks or works you can type JavaScript directly into the browser console (it uses a read-eval-print loop which will affect the page)

==Code==

===nested.js===

<source lang="javascript" line>
var f = function(x) {
function g(y) {
return x + y;
}

return g;
}

var q = f(7);
var h = f(12);

console.log("q(4) = " + q(4));
console.log("h(4) = " + h(4));
</source>

https://jsfiddle.net/Manouchehri/jy5793dd/

===storeLogs.js===

[http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/code/storeLogs.js downloadable version]

<source lang="javascript" line>
// storeLogs.js
//
// node storeLogs.js <logfile>
//
//
// The log messages in the file should be of the format:
// <month> <day of month> <24-hour time in hh:mm:ss> <host> <service name[pid]>: Actual message
//

var mc = require('mongodb').MongoClient;
var fs = require('fs');

var data = fs.readFileSync(process.argv[2], 'utf-8');

var lines = data.split('\n');

var entries = [];

var i, j, entry, field;

for (i=0; i<lines.length; i++) {
if (lines[i] && lines[i] !== '') {
field = lines[i].split(' ');
entry = {};
j = 0;
while (j < field.length) {
if (field[j] === "") {
field.splice(j, 1);
} else {
j++;
}
}
entry.date = field[0] + " " + field[1];
entry.time = field[2];
entry.host = field[3];
entry.service = field[4].slice(0,-1);
entry.message = field.slice(5).join(' ');
entries.push(entry);
}
}

// entry.date = process.argv[2] + " " + process.argv[3];
// entry.time = process.argv[4];
// entry.host = process.argv[5];
// entry.service = process.argv[6].slice(0,-1); // drop the trailing colon
// entry.message = process.argv.slice(7).join(' ');

var db;

var reportInserted = function(err, result) {
if (err) {
throw err;
}

console.log("Inserted the following log record:");
console.log(result.ops);
db.close();
}

var connectCallback = function(err, returnedDB) {
if (err) {
throw err;
}

db = returnedDB;

db.collection('logs').insert(entries, reportInserted);
}

mc.connect('mongodb://localhost/log-demo', connectCallback);
</source>

WebFund 2016W Exam Review

2016-04-11T23:05:40Z

Zero: Added Google Doc link.

==Video==

The video from the review session on April 11, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-examreview-11Apr2016.mp4 is now available].

===Student Notes===

https://docs.google.com/document/d/12aQlwzaVH6Xn9_E6os4xY54KTkZJwKjgKL-qkSfTVGQ/

WebFund 2016W Lecture 23

2016-04-11T17:34:54Z

Zero: Defined CDN.

==Video==

The video from the lecture given on April 5, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec23-05Apr2016.mp4 is now available].

==Notes==

===In class===

<pre>
Lecture 23
----------

Scalability

* You replicate your web application, should be
"embarassingly parallel" (no direct interaction)

* Communication between servers happens through the
backend database

Why not have the web servers talk directly to each other?
- you then have to figure out how to do
synchronization/concurrency right
- that's what databases are for!

So how in the world do you scale up databases?

First answer: use a minimal solution
- only get the functionality that you want

First rule of scalability
- you can't do everything at scale

So, you have to choose what you will do

Why are sacrifices necessary?

latency versus bandwidth

bandwith: bits transferred per second on average
latency: time to get first bit of response after request

Consider a large truck full of hard disks driving
across Canada.
- very, very high bandwidth
- very, very high latency as well!
(2 weeks to get first bit of response)

Ideally, you want high bandwidth and low latency
- bandwidth you get through parallelism
- latency has to be engineered

A "supercomputer" is one with low-latency memory access,
for LOTS of memory
- so it has to have fast interconnects
- thus, accesses to different nodes aren't much
slower than local accesses

Challeng for large web apps is having the database
answer queries with low latency

But some amount of latency is inevitable
- speed of light is finite

So if you want fast access to your webserver worldwide
- you need to replicate across the globe
- be close to your clients

NoSQL databases became popular because of latency
concerns
- you needed to be as fast as possible,
- so strip it to the bone

Use an in-memory key-value store if it is sufficient
- lowest latency
- least functionality

If you have to, use an SQL database
- highest latency
- most functionality

Or use something in between (MongoDB)

Once you choose the type of database, you OPTIMIZE
- minimize I/O and computation required per access
(read or write)
- example: query optimization
- how you form the query
- how database is organized

Count the number of web pages that have the word
"amazing" in them

How?
- first, need a database with a copy of the web pages
- then, you could do linear search through all
of the web pages...

I ask this because a web search is a massive challenge
in query optimization

- need to limit scope as early as possible in query
- organize data so queries are quick to be answered
- precompute as much as possible

The best you can do is table lookup. So have the right
tables ready!

Key tool is making an INDEX
- table of search term and pointers to data

E.g., you have a table of customers sorted by ID
- have an index of names, so a table of names versus
IDs
</pre>

===Student Notes===

====Counting Log Entries====

*Exercise related to Assignment 6: The following is to help you in getting started on adding in single-page functionality for the web app.
**How do we get the data related to the number of log files and the number of entries from the database? Ask the database how many log files are there?
**What do we want to do? Count the number of entries. Use the count method.
***<code>db.logs.count()</code> counts records
**How many distinct values are there?
***<code>db.logs.distinct(“file”)</code> counts how many files we have
**The placeholder information is currently filled in on the server
***How? First go to views. The Jade file already has <code>numfiles</code> and <code>numentries</code> variables which are passed in to the template through the call to <code>render()</code> in the routes. How do we update them?
**What problems do we run in to when using the <code>count()</code> and <code>distinct()</code> methods?
***Reloading the page is one problem
***The real problem is with the callbacks
****In order to do this, we need to do 2 separate database operations to get the values. We can try to combine them, however, this entails nesting one call to the database within the other callback, and therefore is messy.
****Instead, we can make them each as a separate route. Set up the following in index.js:

<code><pre>
router.get(‘/count’, function(req,res){
function reportCount(err,count){
if (err) {
res.send(-1);
}
else {
res.send(count);
}
}

logsCollection.count({}, reportCount);
});
</pre></code>

*A problem with this code is that the count is interpreted as a response code
*We can change the code to fix this:

<code><pre>
router.get(‘/count’, function(req,res){
function reportCount(err,count){
if err {
res.sendStatus(500);}
} else {
res.send({count: count});
}
}

logsCollection.count({}, reportCount);
});
</pre></code>

*And the route for file count:

<code><pre>
router.get(‘/storeFiles’, function(req,res) {
function reportStoredFiles(err,files){
if err {
res.sendStatus(500);
} else{
res.send(files); //to get number of files res.send({count: files.length});
}
}

logsCollection.distinct(“file”, reportStoredFiles);
});
</pre></code>

*This code allows us to get these values to the client
*Currently, we do not have any client-side code to do anything with the values
*We want to somehow update the page to reflect the correct values
*We can do this on the server-side before sending the page to the client but this can be a pain so let’s change this so that we do an AJAX request from the browser and update the DOM
*We will need a client-side script to do this
*We can reference exam-storage to see how to link our scripts in the Jade templates
**In layout.jade, we want to link the jquery script since we will be using it
**From account.jade, we can see where our main client-side script was linked. We will need to do something like this for index.jade
**Make sure that the scripts are stored somewhere in the public directory and that the paths in the links are correct
*We can add some default text into index.jade to display if we do not have the actual file and log entry counts
*Then we need to write our query.js script
**The functions in this script are used to update the number of logs and number of entries shown in the browser. So, when you reload the page after uploading a file, the number of logs and entries also get updated accordingly.
***<code>updateStats()</code> is used to make the AJAX requests to the server to get the updated values
***<code>updatesStatsText()</code> is used to update the DOM with the new values

<code><pre>
$(function() {
var numEntries = 0;
var numFiles = 0;
var stats = $(“#stats”);

function updatesStatsText() {
stats.html(“Currently we have “ + numEntries + “ log entries in “ + numFiles + “log files.”);
}

function updateStats() {
var numUpdated = 0;
$.getJSON(“/ count”,function(v){
numEntries = v.count;
numUpdated++;

if(numUpdated >= 2){
updateStatsText();
}
});

$.getJSON(“/storedFileCount”, function(v){
numFiles = v.count;
numUpdated++;

if(numUpdated >= 2){
updateStatsText(); });
}
});
}

updateStats();
}
</pre></code>

*With the assignment, everything should show up at the bottom of the page.
*This means upating the DOM in a similar fashion to what we have done here

====Scalability====

*When replicating your web application, your code should be “embarrassingly parallel” (no direct interaction)
*Communication between servers happens through the backend database
*Why not have the web servers talk directly to each other?
**You have to figure out how to do synchronization/concurrency right
**That's what databases are for! Let it deal with the problem
*So, how in the world do you scale up databases?
*First answer: use a minimal solution
**Only get the functionality that you want
*Basic/first rule of scalability
**You cant do everything at scale
*So you have to choose what you will do. Have to make some sacrifices
*Why are sacrifices necessary?
*Latency vs Bandwidth
**Bandwidth: bits transferred per second on average
**Latency: time to get first bit of response after request
*Consider a large truck full of hard disks driving across Canada. What’s the bandwidth of this truck? Low bandwidth or high bandwidth?
**It’s very very high bandwidth takes two weeks to travel across the country, but when it arrives, a lot of data has been transferred
**Very very high latency as well! (2 weeks to get first bit of response)
*Ideally you want HIGH BANDWIDTH and LOW LATENCY
**Bandwidth you get through parallelism
**Latency has to be engineered.
*Latency is hard so,
**A “supercoumputer” is one with low latency memory access, for lots of memory.
***So it has to have [https://www.mellanox.com/page/performance_infiniband fast interconnects between the nodes]. Fast meaning low latency, not high bandwidth.
***Thus, accesses to different nodes aren't much slower than local accesses
*Challenge for large web apps is having the database answer queries with low latency
*If a user goes to your website the page should be loaded fast. Even if it's a bit slow, they will be unhappy. So, you need the low latency.
*But some amount of latency is inevitable (laws of physics, Einstein)
**The speed of light is finite. A nanosecond in wire is a short piece of wire. A microsecond a pretty good size spool. Milliseconds is thousand times larger.
**How far do signals go?
***Microprocessors operate on the nanoseconds
***Networks on the milliseconds
***Ping time to get to the other side of the globe, 100s of millisecond
****If it's more than 50 milliseconds or half a second, people will notice
*So if you want fast access to your web server worldwide
**You need to replicate across the globe (this is what a CDN/content delivery network does)
**Be close to your clients
*In general, the more the functionality a database will have, the longer it will take
*NoSQL databases became popular because of latency concerns
**You needed to be as fast as possible
**So strip it to the bone
*Use an in-memory key value store if it is sufficient
**Lowest latency
**Least functionality
*If you have to, use an SQL database
**Higher latency
**Most functionality
*Or use something in between (MongoDB)
*Once you choose the type of database, you optimize
**Minimize I/O and computation require per access (read or write)
**Example: query optimization
**How you form the query
**How the database is organized

 

*Why Google is getting big bucks? How do you query the entire Web?
**Google is good at query optimization
*Google has a copy of everything on the Web. Their Web crawlers do this for them
*How do you make a query of the entire Web?
**Use a very big database!
**The interface to query through Google is just a regular website, it's just working with very large data sets!
*How can we count the number of web pages that have the word that have the word “amazing” in them?
**First you need a database with a copy of all the web pages
**Then you could do linear search through all the of the pages of the web (this is not an efficient solution)
*I ask this because a web search is a massive challenge in query optimization
**We need to limit scope as early as possible in query
**Organize data so queries can be answered quickly
**Precompute as much as possible
*The best you can do is table lookup. So have the right tables ready!
*When you do a Google search, it is picking out answer from answers it already figured out before
*A key tool is making an INDEX
**Table of search terms and pointers to data
**e.g. If you have a table of customers sorted by ID
***Have an index of names, so a table of names vs IDs
**An index is a table that is very short, one column for the indexed attribute and the second for the ID
**Is this what Google is doing? No, they are doing much fancier things but don’t tell anyone because of layers and layers of proprietary stuff
*Microsoft has about 100 millions lines of code, Linux kernel is 20 millions lines of code, Google is estimated at about 2 billion lines of code. This is not surprising, there is a lot of functionality
*Look up MapReduce. You can use it to find all web pages with “amazing”. All you do is you have a cluster of machines with data across them and you tell each machine to count the number of times amazing appears in the web pages you have on that machine and then accumulate altogether to be a single count. It can also handle failures during a query. Data is being stored in special replicated way, so that if there is one particular node that is failing, others can take over the job. It is functional programming at scale.

==Code==

* [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/code/analyzeLogs-filecount.zip analyzeLogs-filecount]

Note this version has node_modules removed; copy this directory from analyzeLogs-sol or run "npm install".

WebFund 2016W Lecture 23

2016-04-11T17:31:32Z

Zero: InfiniBand can be sub-microsecond, as far as I know it's mostly what's being used for supercomputer networking.

==Video==

The video from the lecture given on April 5, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec23-05Apr2016.mp4 is now available].

==Notes==

===In class===

<pre>
Lecture 23
----------

Scalability

* You replicate your web application, should be
"embarassingly parallel" (no direct interaction)

* Communication between servers happens through the
backend database

Why not have the web servers talk directly to each other?
- you then have to figure out how to do
synchronization/concurrency right
- that's what databases are for!

So how in the world do you scale up databases?

First answer: use a minimal solution
- only get the functionality that you want

First rule of scalability
- you can't do everything at scale

So, you have to choose what you will do

Why are sacrifices necessary?

latency versus bandwidth

bandwith: bits transferred per second on average
latency: time to get first bit of response after request

Consider a large truck full of hard disks driving
across Canada.
- very, very high bandwidth
- very, very high latency as well!
(2 weeks to get first bit of response)

Ideally, you want high bandwidth and low latency
- bandwidth you get through parallelism
- latency has to be engineered

A "supercomputer" is one with low-latency memory access,
for LOTS of memory
- so it has to have fast interconnects
- thus, accesses to different nodes aren't much
slower than local accesses

Challeng for large web apps is having the database
answer queries with low latency

But some amount of latency is inevitable
- speed of light is finite

So if you want fast access to your webserver worldwide
- you need to replicate across the globe
- be close to your clients

NoSQL databases became popular because of latency
concerns
- you needed to be as fast as possible,
- so strip it to the bone

Use an in-memory key-value store if it is sufficient
- lowest latency
- least functionality

If you have to, use an SQL database
- highest latency
- most functionality

Or use something in between (MongoDB)

Once you choose the type of database, you OPTIMIZE
- minimize I/O and computation required per access
(read or write)
- example: query optimization
- how you form the query
- how database is organized

Count the number of web pages that have the word
"amazing" in them

How?
- first, need a database with a copy of the web pages
- then, you could do linear search through all
of the web pages...

I ask this because a web search is a massive challenge
in query optimization

- need to limit scope as early as possible in query
- organize data so queries are quick to be answered
- precompute as much as possible

The best you can do is table lookup. So have the right
tables ready!

Key tool is making an INDEX
- table of search term and pointers to data

E.g., you have a table of customers sorted by ID
- have an index of names, so a table of names versus
IDs
</pre>

===Student Notes===

====Counting Log Entries====

*Exercise related to Assignment 6: The following is to help you in getting started on adding in single-page functionality for the web app.
**How do we get the data related to the number of log files and the number of entries from the database? Ask the database how many log files are there?
**What do we want to do? Count the number of entries. Use the count method.
***<code>db.logs.count()</code> counts records
**How many distinct values are there?
***<code>db.logs.distinct(“file”)</code> counts how many files we have
**The placeholder information is currently filled in on the server
***How? First go to views. The Jade file already has <code>numfiles</code> and <code>numentries</code> variables which are passed in to the template through the call to <code>render()</code> in the routes. How do we update them?
**What problems do we run in to when using the <code>count()</code> and <code>distinct()</code> methods?
***Reloading the page is one problem
***The real problem is with the callbacks
****In order to do this, we need to do 2 separate database operations to get the values. We can try to combine them, however, this entails nesting one call to the database within the other callback, and therefore is messy.
****Instead, we can make them each as a separate route. Set up the following in index.js:

<code><pre>
router.get(‘/count’, function(req,res){
function reportCount(err,count){
if (err) {
res.send(-1);
}
else {
res.send(count);
}
}

logsCollection.count({}, reportCount);
});
</pre></code>

*A problem with this code is that the count is interpreted as a response code
*We can change the code to fix this:

<code><pre>
router.get(‘/count’, function(req,res){
function reportCount(err,count){
if err {
res.sendStatus(500);}
} else {
res.send({count: count});
}
}

logsCollection.count({}, reportCount);
});
</pre></code>

*And the route for file count:

<code><pre>
router.get(‘/storeFiles’, function(req,res) {
function reportStoredFiles(err,files){
if err {
res.sendStatus(500);
} else{
res.send(files); //to get number of files res.send({count: files.length});
}
}

logsCollection.distinct(“file”, reportStoredFiles);
});
</pre></code>

*This code allows us to get these values to the client
*Currently, we do not have any client-side code to do anything with the values
*We want to somehow update the page to reflect the correct values
*We can do this on the server-side before sending the page to the client but this can be a pain so let’s change this so that we do an AJAX request from the browser and update the DOM
*We will need a client-side script to do this
*We can reference exam-storage to see how to link our scripts in the Jade templates
**In layout.jade, we want to link the jquery script since we will be using it
**From account.jade, we can see where our main client-side script was linked. We will need to do something like this for index.jade
**Make sure that the scripts are stored somewhere in the public directory and that the paths in the links are correct
*We can add some default text into index.jade to display if we do not have the actual file and log entry counts
*Then we need to write our query.js script
**The functions in this script are used to update the number of logs and number of entries shown in the browser. So, when you reload the page after uploading a file, the number of logs and entries also get updated accordingly.
***<code>updateStats()</code> is used to make the AJAX requests to the server to get the updated values
***<code>updatesStatsText()</code> is used to update the DOM with the new values

<code><pre>
$(function() {
var numEntries = 0;
var numFiles = 0;
var stats = $(“#stats”);

function updatesStatsText() {
stats.html(“Currently we have “ + numEntries + “ log entries in “ + numFiles + “log files.”);
}

function updateStats() {
var numUpdated = 0;
$.getJSON(“/ count”,function(v){
numEntries = v.count;
numUpdated++;

if(numUpdated >= 2){
updateStatsText();
}
});

$.getJSON(“/storedFileCount”, function(v){
numFiles = v.count;
numUpdated++;

if(numUpdated >= 2){
updateStatsText(); });
}
});
}

updateStats();
}
</pre></code>

*With the assignment, everything should show up at the bottom of the page.
*This means upating the DOM in a similar fashion to what we have done here

====Scalability====

*When replicating your web application, your code should be “embarrassingly parallel” (no direct interaction)
*Communication between servers happens through the backend database
*Why not have the web servers talk directly to each other?
**You have to figure out how to do synchronization/concurrency right
**That's what databases are for! Let it deal with the problem
*So, how in the world do you scale up databases?
*First answer: use a minimal solution
**Only get the functionality that you want
*Basic/first rule of scalability
**You cant do everything at scale
*So you have to choose what you will do. Have to make some sacrifices
*Why are sacrifices necessary?
*Latency vs Bandwidth
**Bandwidth: bits transferred per second on average
**Latency: time to get first bit of response after request
*Consider a large truck full of hard disks driving across Canada. What’s the bandwidth of this truck? Low bandwidth or high bandwidth?
**It’s very very high bandwidth takes two weeks to travel across the country, but when it arrives, a lot of data has been transferred
**Very very high latency as well! (2 weeks to get first bit of response)
*Ideally you want HIGH BANDWIDTH and LOW LATENCY
**Bandwidth you get through parallelism
**Latency has to be engineered.
*Latency is hard so,
**A “supercoumputer” is one with low latency memory access, for lots of memory.
***So it has to have [https://www.mellanox.com/page/performance_infiniband fast interconnects between the nodes]. Fast meaning low latency, not high bandwidth.
***Thus, accesses to different nodes aren't much slower than local accesses
*Challenge for large web apps is having the database answer queries with low latency
*If a user goes to your website the page should be loaded fast. Even if it's a bit slow, they will be unhappy. So, you need the low latency.
*But some amount of latency is inevitable (laws of physics, Einstein)
**The speed of light is finite. A nanosecond in wire is a short piece of wire. A microsecond a pretty good size spool. Milliseconds is thousand times larger.
**How far do signals go?
***Microprocessors operate on the nanoseconds
***Networks on the milliseconds
***Ping time to get to the other side of the globe, 100s of millisecond
****If it's more than 50 milliseconds or half a second, people will notice
*So if you want fast access to your web server worldwide
**You need to replicate across the globe
**Be close to your clients
*In general, the more the functionality a database will have, the longer it will take
*NoSQL databases became popular because of latency concerns
**You needed to be as fast as possible
**So strip it to the bone
*Use an in-memory key value store if it is sufficient
**Lowest latency
**Least functionality
*If you have to, use an SQL database
**Higher latency
**Most functionality
*Or use something in between (MongoDB)
*Once you choose the type of database, you optimize
**Minimize I/O and computation require per access (read or write)
**Example: query optimization
**How you form the query
**How the database is organized

 

*Why Google is getting big bucks? How do you query the entire Web?
**Google is good at query optimization
*Google has a copy of everything on the Web. Their Web crawlers do this for them
*How do you make a query of the entire Web?
**Use a very big database!
**The interface to query through Google is just a regular website, it's just working with very large data sets!
*How can we count the number of web pages that have the word that have the word “amazing” in them?
**First you need a database with a copy of all the web pages
**Then you could do linear search through all the of the pages of the web (this is not an efficient solution)
*I ask this because a web search is a massive challenge in query optimization
**We need to limit scope as early as possible in query
**Organize data so queries can be answered quickly
**Precompute as much as possible
*The best you can do is table lookup. So have the right tables ready!
*When you do a Google search, it is picking out answer from answers it already figured out before
*A key tool is making an INDEX
**Table of search terms and pointers to data
**e.g. If you have a table of customers sorted by ID
***Have an index of names, so a table of names vs IDs
**An index is a table that is very short, one column for the indexed attribute and the second for the ID
**Is this what Google is doing? No, they are doing much fancier things but don’t tell anyone because of layers and layers of proprietary stuff
*Microsoft has about 100 millions lines of code, Linux kernel is 20 millions lines of code, Google is estimated at about 2 billion lines of code. This is not surprising, there is a lot of functionality
*Look up MapReduce. You can use it to find all web pages with “amazing”. All you do is you have a cluster of machines with data across them and you tell each machine to count the number of times amazing appears in the web pages you have on that machine and then accumulate altogether to be a single count. It can also handle failures during a query. Data is being stored in special replicated way, so that if there is one particular node that is failing, others can take over the job. It is functional programming at scale.

==Code==

* [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/code/analyzeLogs-filecount.zip analyzeLogs-filecount]

Note this version has node_modules removed; copy this directory from analyzeLogs-sol or run "npm install".

WebFund 2016W Lecture 22

2016-04-11T17:24:27Z

Zero: Minor details.

==Video==

The video for the lecture given on March 31, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec22-31Mar2016.mp4 is now available].

==Notes==

===In Class===

<pre>
Lecture 22
------------
Web Security

Security in isolated systems
- no networking

What is the threat model?

Threat model
- what attacks are you trying to defend against
- what attacks are you NOT trying to defend against

Computer in an isolated room
- ignore physical attacks, e.g. sledgehammers

How to compromise this computer?
- someone bad starts typing
- someone inserts malicious media/device
- parking lot flashdrive attack
(solution: epoxy)
- electromagnetic emissions
(live in a faraday cage)

Computer on a network
- all local attacks
- attacker can send arbitrary network data to system
- attacker can listen in on network traffic
- attacker can change outgoing traffic

Basic defense: network crypto
- encrypt to hide, sign data to protect integrity,
verify authenticity

Who will access?
- authenticated, authorized users
- unauthorized individuals/systems
- attackers (can be either of the above)

How can an attacker be "authorized"?
- social engineering
(can you please reset my password?)
- brute force (try all the passwords)
- technical compromise of credentials (keylogger)
- authorized user turns/is malicious
(insider attack)

And now the web

Who can access your application?
- ANYBODY
- for public web apps, ANYBODY is an authorized user

So just from this we can see that web security is going
to be hard.

The other half of the nightmare: the browser

Ideally, to secure systems you isolate components
- build big walls

Your web browser has walls between pages...but there are
BIG HOLES (on purpose)

- cookies, in particular, are shared
- this is bad because the scope of cookies is
determined by DNS (domain name system)

DNS has almost no security
- there is DNSSEC but nobody uses it, yet at least

You can better protect cookies by using HSTS
- force HTTPS everywhere on a page/domain
- protects against mixes of http/https content and
downgrade attacks

Why do we have cookies in the first place?
- because HTTP was supposed to be stateless

Also...

Cross-site Scripting (XSS) attacks
- not necessarily cross-site, not necesarily scripting
- "web injection attacks"
- see anywhere attacker can inject content into a page
- (ads)
- comments
- social media
- "user-generated content"
- failure to sanitize untrusted input that is inserted
into a web page
(adding a comment to a page)
- otherwise, you can insert arbitrary HTML, CSS,
and JavaScript into the page

Cross-site request forgery
- makes use of cookies automatically being sent
to domain, no matter where the link came from
- can stop using tokens in URLs, or checking
Referer/Origin headers (and a few other ways)

Same-origin policy
- data should only be loaded from the originating
server
- but, you can load almost anything else from
anywhere
- images
- sound
- javascript

So what about JSON?
- when you load it with AJAX, you have same origin
policy
- but if you just treated it as a script...
- and what if you were logged in
- bank sending data as JSON

JSON files are valid JavaScript but they don't specify
the name to give to the data structure
- so you can't access it, unless

To build an object, you have to call the object's
constructor
- why not replace the constructor with our own method?
e.g., override Array
- this is solved in regular browsers for JSON by
only using a clean JavaScript environment for
JSON.parse()

This doesn't help you for code, but who cares about
code, right?
- many websites dynamically generate JavaScript and put
personal information into it

Solution? Don't mix code and data!

WebAssembly
-----------

* Many, many people hate JavaScript
* Lots of code that isn't JavaScript
* But people want everything to run on the web
in a browser

Past efforts at having alternative languages all
had a basic problem
- they couldn't see the DOM
- e.g., Java applets, Flash, Silverlight

And, what about all those other languages?
- C, C++??!
- how about games?

Unreal runs in a web page
- there's WebGL which is pretty fast

So, why can't we compile code and have it run in the
browser?

Two old solutions:
- NaCl (Google)
- asm.js (Mozilla)

They joined up and are now making WebAssembly

This will be good for functionality and bad for
security.
- security won't go well because code isn't
designed for the web
- but it will be cool to play games in a browser at
full speed :-)
</pre>

===Student Notes===

====Administrative====

*There is a study session on April 11 morning.
*Students can come to tutorial next week and consider it as extended office hours

====Web Security====

*There are certain fundamental things of the web that make it very hard to secure
*When considering security, we want to consider a threat model
**What attacks are you trying to defend against?
**What attacks are you NOT trying to defend against?

=====Computer in an isolated room=====

*Ignore physical attacks, e.g. sledgehammers
*If we talking about software level, how can the computer be compromised?
**Someone bad starts typing
**Someone inserts malicious media/device
**Parking lot flashdrive attack (a flashdrive is left in a parking lot for someone to pick up and plug in)
**(solution: epoxy, no one can plug anything in)
**Electromagnetic emissions
***(solution: live in a Faraday cage, almost completely isolated from any EM transmission)

=====Computer on a network=====

*Threats:
**All local attacks (all the physical attack will be there too)
**An attacker can send arbitrary network data to system
**An attacker can listen in on network traffic
**An attacker can change outgoing traffic
**(they can listen anything on the computer, can listen anything going out of the computer, can send arbitrary network data to the computer, can change data and leaving the computer going elsewhere)
*How to protect against that?
*Basic defense: network crypto
**Encrypt to hide, sign data to protect integrity, verify authenticity
*We need to distinguish between who is trying to access the computer
*Who will access?
**Authenticated/authorized users
**Unauthorized individuals/systems
**Attackers (can be either of the above)
*How can an attacker be “authorized” ?
**Social engineering (can you please reset my password?)
**Brute force (try all the passwords)
**Technical compromise of credentials (keylogger)
**Authorized user turns/is malicious (insider attack)(most dangerous, because they are not only authorized, they also know the system)

=====And now the web=====

*Why is the web so nasty?
*Who can access your application?
**ANYBODY
**For most public web apps, ANYBODY is an authorized user
**There are different levels of insider attacks. Just because of you have account on gmail doesn’t mean you have access to the inside system. You don’t have full access, but you have some access, that is potentially dangerous.
*So from this we can see that web security is going to be hard
*The other half of the nightmare is the browser
*Ideally, to secure systems you isolate components
**Build big walls
*Your web browser has walls between pages... but there are BIG HOLES (on purpose)
*The tabs in the browser are not so isolated (e.g Log in to an account on one tab then refresh another tab on the same page... it will be logged in as well). They are connected.
*Cookies are being shared between the pages, it is a shared data source
*This is bad because the scope of cookies is determined by DNS (Domain Name System)
*DNS has almost no security
*There is DNSSEC but nobody uses it, yet at least
**If you want to know more about this, you can find the article on [https://www.usenix.org/blog/security-symposium-talk-measuring-practical-impact-dnssec-deployment usenix.org]
*You can better protect cookies by using HSTS
**This forces HTTPS everywhere on a page/domain
**Protects against mixes of HTTP/HTTPS communications and downgrade attacks
*Why we do we have cookies in the first place?
**Because HTTP was supposed to be stateless
**The cookies are used instead to create a state between the client and server
*If we could get past these problems, would we have proper security?
**Definitely not
*There are other issues...

======Cross-Site Scripting (XSS) Attacks======

*Not necessarily cross-site, and not necessarily scripting
*More like “web injection attacks”
*This can happen anywhere an attacker can inject content into a page
**Ads
**Comments
**Social media
**User-generated content
*Problems can occur when there is a failure to sanitize untrusted input that is inserted into a web page
**This allows users to insert arbitrary HTML, CSS, and JavaScript into the page

======Cross-Site Request Forgery======

*Cross-site scripting is more straight-forward, cross-site request forgery is a little bit weird
*This involves having a link from a page sent by one server cause a request to be sent off to another server
*If you are logged in at the other server, this can cause things to happen which you didn't expect or want to happen
**For example, while logged in to the course wiki, a link from a page on a different website may cause a page to be deleted from the wiki by sending an appropriate request
*This makes use of cookies automatically being sent to a server, no matter where the link came from
*How can we stop this?
**One way is to try to verify where the link came from
**In the browser tools, go to the network tab and check the referer header
***This header indicates where the request came from
**So in principle, to stop a cross-site request forgery, we can check this header in the requests received on the server
***If it came from one of our pages, then we can trust the request
***Otherwise we don’t
**That does not actually work. It kind of works, but not always
**Why? Because the Referer header can often be scrapped by a proxy
**An alternative solution is to randomize your links
**Instead of using a predictable url, stick a random string in the middle of it
***The string is connected to the client's cookie so that the attacker can’t guess it

======Same-Origin Policy======

*Data should only loaded from the originating server but, you can load almost anything else from anywhere
**Images
**Sound
**JavaScript
*So what about JSON?
**when you load it with AJAX, you have the same-origin policy
**But what if you just treated it as a script? (why not just load it in the page?)
**And what if you were logged in?
***Consider a bank sending data as JSON (Could I from another tab just grab your bank account data as a JSON data in the background and see what exactly what you have? It doesn’t work.)
*JSON files are valid JavaScript structure (JSON is a subset of JavaScript)
*JSON files do not specify a name to give to the data structure so you can’t access it by name
*However, to build the object in the first place, its constructor must be called
**Why not replace the constructor with our own method?
***e.g., override Array
**This is solved in regular browsers for JSON by only using a clean JavaScript environment for JSON.parse()
**But what about code?
***This doesn’t help you for code, but who cares about code, right?
**Turns out many websites dynamically generate JavaScript and put personal information into it (different people have different scripts)
***Solution? Don’t mix code and data! (treat the JSON separately, let the code just be loaded on the system, don’t mix it with anything else)

====Web Assembly====

*Many, many people hate JavaScript
**There is a lot of code that isn’t JavaScript but people want everything to run on the web in a browser (JavaScript is the language of the web)
*Past efforts at having alternative languages all had a basic problem
**They couldn’t see the DOM
***e.g., Java applets, Flash, Silverlight
*And, what about all those other languages?
**C, C++?
**How about games?
***[https://www.unrealengine.com/html5/ Unreal Engine runs in a web page] because there’s WebGL which is pretty fast (WebGL is a subset of OpenGL)
*So why can’t we compile code and have it run in the browser?
*Two old solutions:
**PNaCl/NaCl (Google)
***Portable Native Client (PNaCl) is a sandbox for running compiled C and C++ code in the browser efficiently and securely, independent of the user’s operating system or architecture (e.g. x86 or ARM). (NaCl is more or less the same except it only can be used by extensions/plugins and depends on the user's CPU architecture; you can only use an x86 NaCl app on x86.)
**asm.js (Mozilla)
***This uses a compiler that takes arbitrary programs and compiles them to JavaScript
***asm.js is perfectly legal JavaScript, it is tightly restricted JavaScript
***You cannot read asm.js code, it looks like binary
*Google and Mozilla joined up and are now making WebAssembly
**You can compile arbitrary code to run inside of the browser with WebAssembly
**This will be good for functionality and bad for security
***Security won’t go well because the code isn’t designed for the web
***But it will be cool to play games in a browser at full speed

WebFund 2016W Lecture 22

2016-04-11T17:21:27Z

Zero: Added PNaCl vs NaCl.

==Video==

The video for the lecture given on March 31, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec22-31Mar2016.mp4 is now available].

==Notes==

===In Class===

<pre>
Lecture 22
------------
Web Security

Security in isolated systems
- no networking

What is the threat model?

Threat model
- what attacks are you trying to defend against
- what attacks are you NOT trying to defend against

Computer in an isolated room
- ignore physical attacks, e.g. sledgehammers

How to compromise this computer?
- someone bad starts typing
- someone inserts malicious media/device
- parking lot flashdrive attack
(solution: epoxy)
- electromagnetic emissions
(live in a faraday cage)

Computer on a network
- all local attacks
- attacker can send arbitrary network data to system
- attacker can listen in on network traffic
- attacker can change outgoing traffic

Basic defense: network crypto
- encrypt to hide, sign data to protect integrity,
verify authenticity

Who will access?
- authenticated, authorized users
- unauthorized individuals/systems
- attackers (can be either of the above)

How can an attacker be "authorized"?
- social engineering
(can you please reset my password?)
- brute force (try all the passwords)
- technical compromise of credentials (keylogger)
- authorized user turns/is malicious
(insider attack)

And now the web

Who can access your application?
- ANYBODY
- for public web apps, ANYBODY is an authorized user

So just from this we can see that web security is going
to be hard.

The other half of the nightmare: the browser

Ideally, to secure systems you isolate components
- build big walls

Your web browser has walls between pages...but there are
BIG HOLES (on purpose)

- cookies, in particular, are shared
- this is bad because the scope of cookies is
determined by DNS (domain name system)

DNS has almost no security
- there is DNSSEC but nobody uses it, yet at least

You can better protect cookies by using HSTS
- force HTTPS everywhere on a page/domain
- protects against mixes of http/https content and
downgrade attacks

Why do we have cookies in the first place?
- because HTTP was supposed to be stateless

Also...

Cross-site Scripting (XSS) attacks
- not necessarily cross-site, not necesarily scripting
- "web injection attacks"
- see anywhere attacker can inject content into a page
- (ads)
- comments
- social media
- "user-generated content"
- failure to sanitize untrusted input that is inserted
into a web page
(adding a comment to a page)
- otherwise, you can insert arbitrary HTML, CSS,
and JavaScript into the page

Cross-site request forgery
- makes use of cookies automatically being sent
to domain, no matter where the link came from
- can stop using tokens in URLs, or checking
Referer/Origin headers (and a few other ways)

Same-origin policy
- data should only be loaded from the originating
server
- but, you can load almost anything else from
anywhere
- images
- sound
- javascript

So what about JSON?
- when you load it with AJAX, you have same origin
policy
- but if you just treated it as a script...
- and what if you were logged in
- bank sending data as JSON

JSON files are valid JavaScript but they don't specify
the name to give to the data structure
- so you can't access it, unless

To build an object, you have to call the object's
constructor
- why not replace the constructor with our own method?
e.g., override Array
- this is solved in regular browsers for JSON by
only using a clean JavaScript environment for
JSON.parse()

This doesn't help you for code, but who cares about
code, right?
- many websites dynamically generate JavaScript and put
personal information into it

Solution? Don't mix code and data!

WebAssembly
-----------

* Many, many people hate JavaScript
* Lots of code that isn't JavaScript
* But people want everything to run on the web
in a browser

Past efforts at having alternative languages all
had a basic problem
- they couldn't see the DOM
- e.g., Java applets, Flash, Silverlight

And, what about all those other languages?
- C, C++??!
- how about games?

Unreal runs in a web page
- there's WebGL which is pretty fast

So, why can't we compile code and have it run in the
browser?

Two old solutions:
- NaCl (Google)
- asm.js (Mozilla)

They joined up and are now making WebAssembly

This will be good for functionality and bad for
security.
- security won't go well because code isn't
designed for the web
- but it will be cool to play games in a browser at
full speed :-)
</pre>

===Student Notes===

====Administrative====

*There is a study session on April 11 morning.
*Students can come to tutorial next week and consider it as extended office hours

====Web Security====

*There are certain fundamental things of the web that make it very hard to secure
*When considering security, we want to consider a threat model
**What attacks are you trying to defend against?
**What attacks are you NOT trying to defend against?

=====Computer in an isolated room=====

*Ignore physical attacks, e.g. sledgehammers
*If we talking about software level, how can the computer be compromised?
**Someone bad starts typing
**Someone inserts malicious media/device
**Parking lot flashdrive attack (a flashdrive is left in a parking lot for someone to pick up and plug in)
**(solution: epoxy, no one can plug anything in)
**Electromagnetic emissions
***(solution: live in a Faraday cage, almost completely isolated from any EM transmission)

=====Computer on a network=====

*Threats:
**All local attacks (all the physical attack will be there too)
**An attacker can send arbitrary network data to system
**An attacker can listen in on network traffic
**An attacker can change outgoing traffic
**(they can listen anything on the computer, can listen anything going out of the computer, can send arbitrary network data to the computer, can change data and leaving the computer going elsewhere)
*How to protect against that?
*Basic defense: network crypto
**Encrypt to hide, sign data to protect integrity, verify authenticity
*We need to distinguish between who is trying to access the computer
*Who will access?
**Authenticated/authorized users
**Unauthorized individuals/systems
**Attackers (can be either of the above)
*How can an attacker be “authorized” ?
**Social engineering (can you please reset my password?)
**Brute force (try all the passwords)
**Technical compromise of credentials (keylogger)
**Authorized user turns/is malicious (insider attack)(most dangerous, because they are not only authorized, they also know the system)

=====And now the web=====

*Why is the web so nasty?
*Who can access your application?
**ANYBODY
**For most public web apps, ANYBODY is an authorized user
**There are different levels of insider attacks. Just because of you have account on gmail doesn’t mean you have access to the inside system. You don’t have full access, but you have some access, that is potentially dangerous.
*So from this we can see that web security is going to be hard
*The other half of the nightmare is the browser
*Ideally, to secure systems you isolate components
**Build big walls
*Your web browser has walls between pages... but there are BIG HOLES (on purpose)
*The tabs in the browser are not so isolated (e.g Log in to an account on one tab then refresh another tab on the same page... it will be logged in as well). They are connected.
*Cookies are being shared between the pages, it is a shared data source
*This is bad because the scope of cookies is determined by DNS (Domain Name System)
*DNS has almost no security
*There is DNSSEC but nobody uses it, yet at least
**If you want to know more about this, you can find the article on [https://www.usenix.org/blog/security-symposium-talk-measuring-practical-impact-dnssec-deployment usenix.org]
*You can better protect cookies by using HSTS
**This forces HTTPS everywhere on a page/domain
**Protects against mixes of HTTP/HTTPS communications and downgrade attacks
*Why we do we have cookies in the first place?
**Because HTTP was supposed to be stateless
**The cookies are used instead to create a state between the client and server
*If we could get past these problems, would we have proper security?
**Definitely not
*There are other issues...

======Cross-Site Scripting (XSS) Attacks======

*Not necessarily cross-site, and not necessarily scripting
*More like “web injection attacks”
*This can happen anywhere an attacker can inject content into a page
**Ads
**Comments
**Social media
**User-generated content
*Problems can occur when there is a failure to sanitize untrusted input that is inserted into a web page
**This allows users to insert arbitrary HTML, CSS, and JavaScript into the page

======Cross-Site Request Forgery======

*Cross-site scripting is more straight-forward, cross-site request forgery is a little bit weird
*This involves having a link from a page sent by one server cause a request to be sent off to another server
*If you are logged in at the other server, this can cause things to happen which you didn't expect or want to happen
**For example, while logged in to the course wiki, a link from a page on a different website may cause a page to be deleted from the wiki by sending an appropriate request
*This makes use of cookies automatically being sent to a server, no matter where the link came from
*How can we stop this?
**One way is to try to verify where the link came from
**In the browser tools, go to the network tab and check the referer header
***This header indicates where the request came from
**So in principle, to stop a cross-site request forgery, we can check this header in the requests received on the server
***If it came from one of our pages, then we can trust the request
***Otherwise we don’t
**That does not actually work. It kind of works, but not always
**Why? Because the Referer header can often be scrapped by a proxy
**An alternative solution is to randomize your links
**Instead of using a predictable url, stick a random string in the middle of it
***The string is connected to the client's cookie so that the attacker can’t guess it

======Same-Origin Policy======

*Data should only loaded from the originating server but, you can load almost anything else from anywhere
**Images
**Sound
**JavaScript
*So what about JSON?
**when you load it with AJAX, you have the same-origin policy
**But what if you just treated it as a script? (why not just load it in the page?)
**And what if you were logged in?
***Consider a bank sending data as JSON (Could I from another tab just grab your bank account data as a JSON data in the background and see what exactly what you have? It doesn’t work.)
*JSON files are valid JavaScript structure (JSON is a subset of JavaScript)
*JSON files do not specify a name to give to the data structure so you can’t access it by name
*However, to build the object in the first place, its constructor must be called
**Why not replace the constructor with our own method?
***e.g., override Array
**This is solved in regular browsers for JSON by only using a clean JavaScript environment for JSON.parse()
**But what about code?
***This doesn’t help you for code, but who cares about code, right?
**Turns out many websites dynamically generate JavaScript and put personal information into it (different people have different scripts)
***Solution? Don’t mix code and data! (treat the JSON separately, let the code just be loaded on the system, don’t mix it with anything else)

====Web Assembly====

*Many, many people hate JavaScript
**There is a lot of code that isn’t JavaScript but people want everything to run on the web in a browser (JavaScript is the language of the web)
*Past efforts at having alternative languages all had a basic problem
**They couldn’t see the DOM
***e.g., Java applets, Flash, Silverlight
*And, what about all those other languages?
**C,C++?
**How about games?
***Unreal runs in a web page because there’s WebGL which is pretty fast (WebGL is a subset of openGL)
*So why can’t we compile code and have it run in the browser?
*Two old solutions:
**PNaCl/NaCl (Google)
***Portable Native Client (PNaCl) is a sandbox for running compiled C and C++ code in the browser efficiently and securely, independent of the user’s operating system or architecture (e.g. x86 or ARM). (NaCl is more or less the same except it only can be used by extensions/plugins and depends on the user's CPU architecture; you can only use an x86 NaCl app on x86.)
**asm.js (Mozilla)
***This uses a compiler that takes arbitrary programs and compiles them to JavaScript
***asm.js is perfectly legal JavaScript, it is tightly restricted JavaScript
***You cannot read asm.js code, it looks like binary
*Google and Mozilla joined up and are now making WebAssembly
**You can compile arbitrary code to run inside of the browser with WebAssembly
**This will be good for functionality and bad for security
***Security won’t go well because the code isn’t designed for the web
***But it will be cool to play games in a browser at full speed

WebFund 2016W Lecture 22

2016-04-11T17:14:11Z

Zero: I believe this is the correct URL?

==Video==

The video for the lecture given on March 31, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec22-31Mar2016.mp4 is now available].

==Notes==

===In Class===

<pre>
Lecture 22
------------
Web Security

Security in isolated systems
- no networking

What is the threat model?

Threat model
- what attacks are you trying to defend against
- what attacks are you NOT trying to defend against

Computer in an isolated room
- ignore physical attacks, e.g. sledgehammers

How to compromise this computer?
- someone bad starts typing
- someone inserts malicious media/device
- parking lot flashdrive attack
(solution: epoxy)
- electromagnetic emissions
(live in a faraday cage)

Computer on a network
- all local attacks
- attacker can send arbitrary network data to system
- attacker can listen in on network traffic
- attacker can change outgoing traffic

Basic defense: network crypto
- encrypt to hide, sign data to protect integrity,
verify authenticity

Who will access?
- authenticated, authorized users
- unauthorized individuals/systems
- attackers (can be either of the above)

How can an attacker be "authorized"?
- social engineering
(can you please reset my password?)
- brute force (try all the passwords)
- technical compromise of credentials (keylogger)
- authorized user turns/is malicious
(insider attack)

And now the web

Who can access your application?
- ANYBODY
- for public web apps, ANYBODY is an authorized user

So just from this we can see that web security is going
to be hard.

The other half of the nightmare: the browser

Ideally, to secure systems you isolate components
- build big walls

Your web browser has walls between pages...but there are
BIG HOLES (on purpose)

- cookies, in particular, are shared
- this is bad because the scope of cookies is
determined by DNS (domain name system)

DNS has almost no security
- there is DNSSEC but nobody uses it, yet at least

You can better protect cookies by using HSTS
- force HTTPS everywhere on a page/domain
- protects against mixes of http/https content and
downgrade attacks

Why do we have cookies in the first place?
- because HTTP was supposed to be stateless

Also...

Cross-site Scripting (XSS) attacks
- not necessarily cross-site, not necesarily scripting
- "web injection attacks"
- see anywhere attacker can inject content into a page
- (ads)
- comments
- social media
- "user-generated content"
- failure to sanitize untrusted input that is inserted
into a web page
(adding a comment to a page)
- otherwise, you can insert arbitrary HTML, CSS,
and JavaScript into the page

Cross-site request forgery
- makes use of cookies automatically being sent
to domain, no matter where the link came from
- can stop using tokens in URLs, or checking
Referer/Origin headers (and a few other ways)

Same-origin policy
- data should only be loaded from the originating
server
- but, you can load almost anything else from
anywhere
- images
- sound
- javascript

So what about JSON?
- when you load it with AJAX, you have same origin
policy
- but if you just treated it as a script...
- and what if you were logged in
- bank sending data as JSON

JSON files are valid JavaScript but they don't specify
the name to give to the data structure
- so you can't access it, unless

To build an object, you have to call the object's
constructor
- why not replace the constructor with our own method?
e.g., override Array
- this is solved in regular browsers for JSON by
only using a clean JavaScript environment for
JSON.parse()

This doesn't help you for code, but who cares about
code, right?
- many websites dynamically generate JavaScript and put
personal information into it

Solution? Don't mix code and data!

WebAssembly
-----------

* Many, many people hate JavaScript
* Lots of code that isn't JavaScript
* But people want everything to run on the web
in a browser

Past efforts at having alternative languages all
had a basic problem
- they couldn't see the DOM
- e.g., Java applets, Flash, Silverlight

And, what about all those other languages?
- C, C++??!
- how about games?

Unreal runs in a web page
- there's WebGL which is pretty fast

So, why can't we compile code and have it run in the
browser?

Two old solutions:
- NaCl (Google)
- asm.js (Mozilla)

They joined up and are now making WebAssembly

This will be good for functionality and bad for
security.
- security won't go well because code isn't
designed for the web
- but it will be cool to play games in a browser at
full speed :-)
</pre>

===Student Notes===

====Administrative====

*There is a study session on April 11 morning.
*Students can come to tutorial next week and consider it as extended office hours

====Web Security====

*There are certain fundamental things of the web that make it very hard to secure
*When considering security, we want to consider a threat model
**What attacks are you trying to defend against?
**What attacks are you NOT trying to defend against?

=====Computer in an isolated room=====

*Ignore physical attacks, e.g. sledgehammers
*If we talking about software level, how can the computer be compromised?
**Someone bad starts typing
**Someone inserts malicious media/device
**Parking lot flashdrive attack (a flashdrive is left in a parking lot for someone to pick up and plug in)
**(solution: epoxy, no one can plug anything in)
**Electromagnetic emissions
***(solution: live in a Faraday cage, almost completely isolated from any EM transmission)

=====Computer on a network=====

*Threats:
**All local attacks (all the physical attack will be there too)
**An attacker can send arbitrary network data to system
**An attacker can listen in on network traffic
**An attacker can change outgoing traffic
**(they can listen anything on the computer, can listen anything going out of the computer, can send arbitrary network data to the computer, can change data and leaving the computer going elsewhere)
*How to protect against that?
*Basic defense: network crypto
**Encrypt to hide, sign data to protect integrity, verify authenticity
*We need to distinguish between who is trying to access the computer
*Who will access?
**Authenticated/authorized users
**Unauthorized individuals/systems
**Attackers (can be either of the above)
*How can an attacker be “authorized” ?
**Social engineering (can you please reset my password?)
**Brute force (try all the passwords)
**Technical compromise of credentials (keylogger)
**Authorized user turns/is malicious (insider attack)(most dangerous, because they are not only authorized, they also know the system)

=====And now the web=====

*Why is the web so nasty?
*Who can access your application?
**ANYBODY
**For most public web apps, ANYBODY is an authorized user
**There are different levels of insider attacks. Just because of you have account on gmail doesn’t mean you have access to the inside system. You don’t have full access, but you have some access, that is potentially dangerous.
*So from this we can see that web security is going to be hard
*The other half of the nightmare is the browser
*Ideally, to secure systems you isolate components
**Build big walls
*Your web browser has walls between pages... but there are BIG HOLES (on purpose)
*The tabs in the browser are not so isolated (e.g Log in to an account on one tab then refresh another tab on the same page... it will be logged in as well). They are connected.
*Cookies are being shared between the pages, it is a shared data source
*This is bad because the scope of cookies is determined by DNS (Domain Name System)
*DNS has almost no security
*There is DNSSEC but nobody uses it, yet at least
**If you want to know more about this, you can find the article on [https://www.usenix.org/blog/security-symposium-talk-measuring-practical-impact-dnssec-deployment usenix.org]
*You can better protect cookies by using HSTS
**This forces HTTPS everywhere on a page/domain
**Protects against mixes of HTTP/HTTPS communications and downgrade attacks
*Why we do we have cookies in the first place?
**Because HTTP was supposed to be stateless
**The cookies are used instead to create a state between the client and server
*If we could get past these problems, would we have proper security?
**Definitely not
*There are other issues...

======Cross-Site Scripting (XSS) Attacks======

*Not necessarily cross-site, and not necessarily scripting
*More like “web injection attacks”
*This can happen anywhere an attacker can inject content into a page
**Ads
**Comments
**Social media
**User-generated content
*Problems can occur when there is a failure to sanitize untrusted input that is inserted into a web page
**This allows users to insert arbitrary HTML, CSS, and JavaScript into the page

======Cross-Site Request Forgery======

*Cross-site scripting is more straight-forward, cross-site request forgery is a little bit weird
*This involves having a link from a page sent by one server cause a request to be sent off to another server
*If you are logged in at the other server, this can cause things to happen which you didn't expect or want to happen
**For example, while logged in to the course wiki, a link from a page on a different website may cause a page to be deleted from the wiki by sending an appropriate request
*This makes use of cookies automatically being sent to a server, no matter where the link came from
*How can we stop this?
**One way is to try to verify where the link came from
**In the browser tools, go to the network tab and check the referer header
***This header indicates where the request came from
**So in principle, to stop a cross-site request forgery, we can check this header in the requests received on the server
***If it came from one of our pages, then we can trust the request
***Otherwise we don’t
**That does not actually work. It kind of works, but not always
**Why? Because the Referer header can often be scrapped by a proxy
**An alternative solution is to randomize your links
**Instead of using a predictable url, stick a random string in the middle of it
***The string is connected to the client's cookie so that the attacker can’t guess it

======Same-Origin Policy======

*Data should only loaded from the originating server but, you can load almost anything else from anywhere
**Images
**Sound
**JavaScript
*So what about JSON?
**when you load it with AJAX, you have the same-origin policy
**But what if you just treated it as a script? (why not just load it in the page?)
**And what if you were logged in?
***Consider a bank sending data as JSON (Could I from another tab just grab your bank account data as a JSON data in the background and see what exactly what you have? It doesn’t work.)
*JSON files are valid JavaScript structure (JSON is a subset of JavaScript)
*JSON files do not specify a name to give to the data structure so you can’t access it by name
*However, to build the object in the first place, its constructor must be called
**Why not replace the constructor with our own method?
***e.g., override Array
**This is solved in regular browsers for JSON by only using a clean JavaScript environment for JSON.parse()
**But what about code?
***This doesn’t help you for code, but who cares about code, right?
**Turns out many websites dynamically generate JavaScript and put personal information into it (different people have different scripts)
***Solution? Don’t mix code and data! (treat the JSON separately, let the code just be loaded on the system, don’t mix it with anything else)

====Web Assembly====

*Many, many people hate JavaScript
**There is a lot of code that isn’t JavaScript but people want everything to run on the web in a browser (JavaScript is the language of the web)
*Past efforts at having alternative languages all had a basic problem
**They couldn’t see the DOM
***e.g., Java applets, Flash, Silverlight
*And, what about all those other languages?
**C,C++?
**How about games?
***Unreal runs in a web page because there’s WebGL which is pretty fast (WebGL is a subset of openGL)
*So why can’t we compile code and have it run in the browser?
*Two old solutions:
**NaCl (Google)
***Native Client is a sandbox for running compiled C and C++ code in the browser efficiently and securely, independent of the user’s operating system
**asm.js (Mozilla)
***This uses a compiler that takes arbitrary programs and compiles them to JavaScript
***asm.js is perfectly legal JavaScript, it is tightly restricted JavaScript
***You cannot read asm.js code, it looks like binary
*Google and Mozilla joined up and are now making WebAssembly
**You can compile arbitrary code to run inside of the browser with WebAssembly
**This will be good for functionality and bad for security
***Security won’t go well because the code isn’t designed for the web
***But it will be cool to play games in a browser at full speed

WebFund 2016W Lecture 22

2016-04-11T17:11:03Z

Zero: Typo.

==Video==

The video for the lecture given on March 31, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec22-31Mar2016.mp4 is now available].

==Notes==

===In Class===

<pre>
Lecture 22
------------
Web Security

Security in isolated systems
- no networking

What is the threat model?

Threat model
- what attacks are you trying to defend against
- what attacks are you NOT trying to defend against

Computer in an isolated room
- ignore physical attacks, e.g. sledgehammers

How to compromise this computer?
- someone bad starts typing
- someone inserts malicious media/device
- parking lot flashdrive attack
(solution: epoxy)
- electromagnetic emissions
(live in a faraday cage)

Computer on a network
- all local attacks
- attacker can send arbitrary network data to system
- attacker can listen in on network traffic
- attacker can change outgoing traffic

Basic defense: network crypto
- encrypt to hide, sign data to protect integrity,
verify authenticity

Who will access?
- authenticated, authorized users
- unauthorized individuals/systems
- attackers (can be either of the above)

How can an attacker be "authorized"?
- social engineering
(can you please reset my password?)
- brute force (try all the passwords)
- technical compromise of credentials (keylogger)
- authorized user turns/is malicious
(insider attack)

And now the web

Who can access your application?
- ANYBODY
- for public web apps, ANYBODY is an authorized user

So just from this we can see that web security is going
to be hard.

The other half of the nightmare: the browser

Ideally, to secure systems you isolate components
- build big walls

Your web browser has walls between pages...but there are
BIG HOLES (on purpose)

- cookies, in particular, are shared
- this is bad because the scope of cookies is
determined by DNS (domain name system)

DNS has almost no security
- there is DNSSEC but nobody uses it, yet at least

You can better protect cookies by using HSTS
- force HTTPS everywhere on a page/domain
- protects against mixes of http/https content and
downgrade attacks

Why do we have cookies in the first place?
- because HTTP was supposed to be stateless

Also...

Cross-site Scripting (XSS) attacks
- not necessarily cross-site, not necesarily scripting
- "web injection attacks"
- see anywhere attacker can inject content into a page
- (ads)
- comments
- social media
- "user-generated content"
- failure to sanitize untrusted input that is inserted
into a web page
(adding a comment to a page)
- otherwise, you can insert arbitrary HTML, CSS,
and JavaScript into the page

Cross-site request forgery
- makes use of cookies automatically being sent
to domain, no matter where the link came from
- can stop using tokens in URLs, or checking
Referer/Origin headers (and a few other ways)

Same-origin policy
- data should only be loaded from the originating
server
- but, you can load almost anything else from
anywhere
- images
- sound
- javascript

So what about JSON?
- when you load it with AJAX, you have same origin
policy
- but if you just treated it as a script...
- and what if you were logged in
- bank sending data as JSON

JSON files are valid JavaScript but they don't specify
the name to give to the data structure
- so you can't access it, unless

To build an object, you have to call the object's
constructor
- why not replace the constructor with our own method?
e.g., override Array
- this is solved in regular browsers for JSON by
only using a clean JavaScript environment for
JSON.parse()

This doesn't help you for code, but who cares about
code, right?
- many websites dynamically generate JavaScript and put
personal information into it

Solution? Don't mix code and data!

WebAssembly
-----------

* Many, many people hate JavaScript
* Lots of code that isn't JavaScript
* But people want everything to run on the web
in a browser

Past efforts at having alternative languages all
had a basic problem
- they couldn't see the DOM
- e.g., Java applets, Flash, Silverlight

And, what about all those other languages?
- C, C++??!
- how about games?

Unreal runs in a web page
- there's WebGL which is pretty fast

So, why can't we compile code and have it run in the
browser?

Two old solutions:
- NaCl (Google)
- asm.js (Mozilla)

They joined up and are now making WebAssembly

This will be good for functionality and bad for
security.
- security won't go well because code isn't
designed for the web
- but it will be cool to play games in a browser at
full speed :-)
</pre>

===Student Notes===

====Administrative====

*There is a study session on April 11 morning.
*Students can come to tutorial next week and consider it as extended office hours

====Web Security====

*There are certain fundamental things of the web that make it very hard to secure
*When considering security, we want to consider a threat model
**What attacks are you trying to defend against?
**What attacks are you NOT trying to defend against?

=====Computer in an isolated room=====

*Ignore physical attacks, e.g. sledgehammers
*If we talking about software level, how can the computer be compromised?
**Someone bad starts typing
**Someone inserts malicious media/device
**Parking lot flashdrive attack (a flashdrive is left in a parking lot for someone to pick up and plug in)
**(solution: epoxy, no one can plug anything in)
**Electromagnetic emissions
***(solution: live in a Faraday cage, almost completely isolated from any EM transmission)

=====Computer on a network=====

*Threats:
**All local attacks (all the physical attack will be there too)
**An attacker can send arbitrary network data to system
**An attacker can listen in on network traffic
**An attacker can change outgoing traffic
**(they can listen anything on the computer, can listen anything going out of the computer, can send arbitrary network data to the computer, can change data and leaving the computer going elsewhere)
*How to protect against that?
*Basic defense: network crypto
**Encrypt to hide, sign data to protect integrity, verify authenticity
*We need to distinguish between who is trying to access the computer
*Who will access?
**Authenticated/authorized users
**Unauthorized individuals/systems
**Attackers (can be either of the above)
*How can an attacker be “authorized” ?
**Social engineering (can you please reset my password?)
**Brute force (try all the passwords)
**Technical compromise of credentials (keylogger)
**Authorized user turns/is malicious (insider attack)(most dangerous, because they are not only authorized, they also know the system)

=====And now the web=====

*Why is the web so nasty?
*Who can access your application?
**ANYBODY
**For most public web apps, ANYBODY is an authorized user
**There are different levels of insider attacks. Just because of you have account on gmail doesn’t mean you have access to the inside system. You don’t have full access, but you have some access, that is potentially dangerous.
*So from this we can see that web security is going to be hard
*The other half of the nightmare is the browser
*Ideally, to secure systems you isolate components
**Build big walls
*Your web browser has walls between pages... but there are BIG HOLES (on purpose)
*The tabs in the browser are not so isolated (e.g Log in to an account on one tab then refresh another tab on the same page... it will be logged in as well). They are connected.
*Cookies are being shared between the pages, it is a shared data source
*This is bad because the scope of cookies is determined by DNS (Domain Name System)
*DNS has almost no security
*There is DNSSEC but nobody uses it, yet at least
**If you want to know more about this, you can find the article on [usenix.org usenix.org]
*You can better protect cookies by using HSTS
**This forces HTTPS everywhere on a page/domain
**Protects against mixes of HTTP/HTTPS communications and downgrade attacks
*Why we do we have cookies in the first place?
**Because HTTP was supposed to be stateless
**The cookies are used instead to create a state between the client and server
*If we could get past these problems, would we have proper security?
**Definitely not
*There are other issues...

======Cross-Site Scripting (XSS) Attacks======

*Not necessarily cross-site, and not necessarily scripting
*More like “web injection attacks”
*This can happen anywhere an attacker can inject content into a page
**Ads
**Comments
**Social media
**User-generated content
*Problems can occur when there is a failure to sanitize untrusted input that is inserted into a web page
**This allows users to insert arbitrary HTML, CSS, and JavaScript into the page

======Cross-Site Request Forgery======

*Cross-site scripting is more straight-forward, cross-site request forgery is a little bit weird
*This involves having a link from a page sent by one server cause a request to be sent off to another server
*If you are logged in at the other server, this can cause things to happen which you didn't expect or want to happen
**For example, while logged in to the course wiki, a link from a page on a different website may cause a page to be deleted from the wiki by sending an appropriate request
*This makes use of cookies automatically being sent to a server, no matter where the link came from
*How can we stop this?
**One way is to try to verify where the link came from
**In the browser tools, go to the network tab and check the referer header
***This header indicates where the request came from
**So in principle, to stop a cross-site request forgery, we can check this header in the requests received on the server
***If it came from one of our pages, then we can trust the request
***Otherwise we don’t
**That does not actually work. It kind of works, but not always
**Why? Because the Referer header can often be scrapped by a proxy
**An alternative solution is to randomize your links
**Instead of using a predictable url, stick a random string in the middle of it
***The string is connected to the client's cookie so that the attacker can’t guess it

======Same-Origin Policy======

*Data should only loaded from the originating server but, you can load almost anything else from anywhere
**Images
**Sound
**JavaScript
*So what about JSON?
**when you load it with AJAX, you have the same-origin policy
**But what if you just treated it as a script? (why not just load it in the page?)
**And what if you were logged in?
***Consider a bank sending data as JSON (Could I from another tab just grab your bank account data as a JSON data in the background and see what exactly what you have? It doesn’t work.)
*JSON files are valid JavaScript structure (JSON is a subset of JavaScript)
*JSON files do not specify a name to give to the data structure so you can’t access it by name
*However, to build the object in the first place, its constructor must be called
**Why not replace the constructor with our own method?
***e.g., override Array
**This is solved in regular browsers for JSON by only using a clean JavaScript environment for JSON.parse()
**But what about code?
***This doesn’t help you for code, but who cares about code, right?
**Turns out many websites dynamically generate JavaScript and put personal information into it (different people have different scripts)
***Solution? Don’t mix code and data! (treat the JSON separately, let the code just be loaded on the system, don’t mix it with anything else)

====Web Assembly====

*Many, many people hate JavaScript
**There is a lot of code that isn’t JavaScript but people want everything to run on the web in a browser (JavaScript is the language of the web)
*Past efforts at having alternative languages all had a basic problem
**They couldn’t see the DOM
***e.g., Java applets, Flash, Silverlight
*And, what about all those other languages?
**C,C++?
**How about games?
***Unreal runs in a web page because there’s WebGL which is pretty fast (WebGL is a subset of openGL)
*So why can’t we compile code and have it run in the browser?
*Two old solutions:
**NaCl (Google)
***Native Client is a sandbox for running compiled C and C++ code in the browser efficiently and securely, independent of the user’s operating system
**asm.js (Mozilla)
***This uses a compiler that takes arbitrary programs and compiles them to JavaScript
***asm.js is perfectly legal JavaScript, it is tightly restricted JavaScript
***You cannot read asm.js code, it looks like binary
*Google and Mozilla joined up and are now making WebAssembly
**You can compile arbitrary code to run inside of the browser with WebAssembly
**This will be good for functionality and bad for security
***Security won’t go well because the code isn’t designed for the web
***But it will be cool to play games in a browser at full speed

WebFund 2016W Lecture 11

2016-02-23T20:54:53Z

Zero: Added more in-class notes.

==Notes==

===In-class Notes===

<pre>
Lecture 11
----------

Databases

/var/spool/mail
- where email is stored
- one file per user

my mail inbox in /var/spool/mail/soma

Who accesses this file?
- mail client
- mail server

What happens if the mail server delivers mail while I'm going
through my email (deleting, filing)
- unless you lock it, you lose mail, mail client will overwrite
inbox and delete new messages
- solution: lock the inbox
create a soma.lock file
(inside it has the process ID of the accessing process)

This is a bad idea that does not scale
Instead, use a database
- concurrent access
- data persistence

What kind of database?
- key-value stores
like a filesystem for very small files,
with search
- document store <-- we'll use this
values have structure
can index on values
- relational database
sets of tables
tables have rows and columns
relations between tables
example: names and addresses
- rows are individual people
- columns are first name, last name, address, city, etc.
relation example: customers and purchases
- each purchase (invoice) has item purchased, price, and
customer ID
- customer table connects customer ID and name, address, etc

We're going to use MongoDB, a document store, not a relational
database.
- stores JSON-like documents
- scriptable in JavaScript

If we were to use a relational database, we'd have to learn SQL
- Structured Query Language
- many man implementations
Oracle, MySQL (MariaDB), PostgreSQL, MS SQL Server, SQLite, etc
- you mainly need a relational database to handle transactions

Transaction
- what if a change to a database involves multiple tables?
- a transaction ensures all tables are changed or none are

Classic example: airline tickets

Tables:
- seat reservations
- payment
- customer info

When you buy a ticket, you want to change all three

A transaction ensures atomic behavior
all happen or none happen

MongoDB Relational DB
Document => row
Collection => table

Web Architecture
----------------

web browser (interprets HTML, CSS, JavaScript)

|
| HTTP
|

web server (runs arbitrary code, in this class JavaScript in node)

|
| SQL or MongoDB protocol
|

database (persistent store, allows concurrent access safely)
</pre>

WebFund 2016W Lecture 11

2016-02-23T20:20:24Z

Zero: Added more in-class notes.

WebFund 2016W Lecture 11

2016-02-23T19:56:14Z

Zero: Added partial in class notes.

Fundamentals of Web Applications (Winter 2016)

2016-02-03T00:01:09Z

Zero: As stated in the outline, a wiki account is no longer required.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class)

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|Exam forms]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 31 (extended)

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity. Keep in mind that the browser is likely out of date and you should not log into any important services (e.g., Gmail) due to missing security patches.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

====Working directly off network drive (sshfs)====

If you don't want to bother running save2406 and restore2406 all the time, you can optionally mount the network drive.

sudo apt-get install sshfs # You may already have this package
mkdir ~/network-drive/ # Pick any name you want.
sshfs -o reconnect,transform_symlinks ~/network-drive/ <SCS username>@access.scs.carleton.ca:/

At this point, you should see all of your files in ~/network-drive/COMP2406/.

Note: There is no concept of "restore" or undo. If you delete or overwrite a file, it's gone.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova virtual appliance file] and import into [http://www.virtualbox.org VirtualBox] or VMware Workstation/Fusion ([https://secure.scs.carleton.ca/vmware/ free to Carleton SCS students]). The SHA1 hash of this file is:

97fc4de4544505178b580d27487085a20961e91e [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova COMP 2406, Winter 2016.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If your virtualization platform is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in native Windows. Here are some tips on getting things running.

* You can install the [https://nodejs.org/en/download/ official Node.js package] to get basic node and npm functionality.
* See [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].
* [https://atom.io/ Atom] is a another popular free GUI text editor.

===Configuring Mac OS X===

It is possible, and recommended, to configure Node and mongoDB to run on your Mac.

====Homebrew method (recommended)====
Homebrew is a package manager for OS X that works similarly to Linux-style package managers like <tt>apt</tt>. Homebrew provides access to thousands of Linux and BSD packages to install natively on your Mac. You have a perfectly capable UNIX system in front of you. Make good use of it!

* Install [http://brew.sh/ Homebrew] (copy the installation text on that page to Terminal.app)
* Install Node and mongoDB

Run the following commands in your terminal:

brew update
brew install node
brew install mongo

That's it!

====Manual method====
Alternatively, you can download and install Node and mongo directly.

* Download and install the ''Mac OS X Installer'' from [https://nodejs.org/en/download/stable/ the download page].
* Download the binaries from [https://www.mongodb.org/downloads#production MongoDB's download page]
* Follow [https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-manually these instructions]

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

You will send your notes to Lee to verify that your notes are of sufficient quality to merit the extra credit; this may require a few rounds of revisions. However, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

Once your notes meet the guidelines, Lee will post the notes for you.

WebFund 2016W Lecture 8

2016-02-02T20:30:14Z

Zero: Added some notes. Not the greatest, somebody else should actually finish them.

==Student Notes==

===Agenda===

* Spoke/explained the solutions for [[WebFund 2016W: Assignment 2|assignment 2]].
* Answered questions about the upcoming [[WebFund 2016W: Assignment 3|assignment 3]].

===Topics (from assignment 2 & 3)===

* What does the regular expression on line 92 of tinywebserver.js do? (Answer: Prevents breaking out of the current directory by using <tt>..</tt> and <tt>~</tt>)
* What does path.normalize do? (Answer: Tries to fix a malformed path. e.g. <tt>/home//dave/</tt> becomes <tt>/home/dave/</tt>)
* What is Jade? (Answer: Templating language.)

User:Zero

2016-01-29T01:55:21Z

Zero: Use HTTPS.

[https://www.google.com/recaptcha/mailhide/d?k=01sDNx0w0evmNHUjPrRHXNDA==&c=EUckNBLbP39p5lu7CVKLHL9A3qmF8PeG7RO1TesVKY1-23rQ9oQJE3lGAEDqhIiD Click here] for my email address if you want to send me any comments/questions.

User:Zero

2016-01-29T01:52:56Z

Zero: Added page.

[http://www.google.com/recaptcha/mailhide/d?k=01sDNx0w0evmNHUjPrRHXNDA==&c=EUckNBLbP39p5lu7CVKLHL9A3qmF8PeG7RO1TesVKY1-23rQ9oQJE3lGAEDqhIiD Click here] for my email address if you want to send me any comments/questions.

WebFund 2016W Lecture 7

2016-01-28T21:27:35Z

Zero: No in-class notes this time.

==Video==

The video for the lecture given on January 28, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec01-07Jan2016.mp4 is now available].

==Notes==

===In-class Poll===

https://pollev.com/anilsomayaji565

WebFund 2016W Lecture 7

2016-01-28T19:47:55Z

Zero: Added some of the in-class notes.

==Notes==

===In-class Poll===

https://pollev.com/anilsomayaji565

===In-class Notes===

<pre>
Lecture #7
---------

</pre>

WebFund 2016W Lecture 6

2016-01-26T20:52:23Z

Zero: Added more in-class notes.

==Notes==

===In-class Notes===

<pre>
Lecture #6
---------

PLAN
* Assignment 2
- aliases
- headers
- other clarifications?

* Objects
- associative hashes with inheritance
- prototypes using Object.create
- "global" object
- "Object" object
- Object.prototype
- hasOwnProperty and Object.hasOwnProperty

* Function calling and "this"
- functions in regular scope
- methods
- constructors with new
- call/apply
- functions in global scope

* form demo!

------------------

If a function starts with a capital letter, it is an object
constructor that you call with new.
- why? because it keeps JavaScript programmers sort of sane

if (x.a) { blah} instead do

if (x.hasOwnProperty(a) && x.a) better, but

if (hasOwnProperty.call(x,a) && x.a) is even better

Why not tinywebserver?

* templates: otherwise, have to manually generate web pages using
variable data
* routing: otherwise, you'll be using regexp's to match each
incoming URL and then call the right function

</pre>

WebFund 2016W Lecture 6

2016-01-26T19:44:59Z

Zero: Typo.

WebFund 2016W Lecture 6

2016-01-26T19:41:33Z

Zero: Added in-class notes from the beginning of the lecture.

WebFund 2016W Lecture 4

2016-01-22T03:04:37Z

Zero: Added nosniff header.

==Video==

The video for the lecture given on January 19, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec04-19Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture 4
---------
Web servers

What does a web server do?
Listens on port 80, 443, or other port
INPUT: HTTP requests
OUTPUT: HTTP responses & requested documents

tinywebserver.js
- written in JavaScript (Node)
- can only serve files
- understands only a few file types

So tinywebserver is how web servers worked in 1995

What changed since then is now web servers run code to generate responses

Why use frameworks?
- because we don't want to build up HTML from scratch
- and, we want logic that is abstracted away
- and, we want routing
- have someone else decide which function to call in response to requests

But for this week, bare bones

Debugging

</pre>

===Student Notes===
====Announcements====
*Assignment 1 due tomorrow (now pushed back to Thursday before class)
*Basics for tutorial 2 and assignment 2 are up!

====Web Servers====
*What does a web server do?
**Listens on port 80 (usually HTTP), 443 (usually HTTPS) or some other port.
***If the web server is listening on a non-standard port, you must specify the port in the URL (using something like “localhost:8080”)
**Input: HTTP requests
**Output: HTTP responses with requested documents
***HTTP responses always have a response code
****200: OK, used for successful request
****404: Not Found
**In other words it services requests
*Need root privileges to listen on ports <~1024
**Do not do development as root if you can, it is risky!
*Popular Web Servers: Apache, NginX

====Tiny Web Server====
*Written in JavaScript(Node)
*Can only serve files! (most modern servers can do more)
*Understands limited file types
*Similar to web servers from 1995
*To try to understand what the code is doing, we can start by looking at the bottom of the file

=====http.createServer()=====
*Apart from the loading of some modules and the definitions of some objects and functions, the first real code to run in this script is the <code>createServer()</code> method
*We are calling a method of the <code>http</code> object (loaded from a module) in order to create an object which will act as the server (which we then store in the <code>server</code> variable)
*This method takes a single argument, <code>request_handler</code>
**<code>request_handler</code> is a function which is intended to handler incoming requests
**By providing this function as the argument, we are telling the server to call the function every time a request comes in

=====server.listen()=====
*Here we are calling the <code>listen()</code> method of the <code>server</code> object that was just created
*This tells the server to start listening for incoming requests
*The first two arguments are telling the server where to listen (on port 8080 of localhost)
**This information came from the <code>options</code> object which was defined at the top of the file
*The third argument is a callback function
**This will run once the server has started listening
**It simply prints a message in the console to indicate that the server is ready

=====request_handler()=====
*This function gets called every time a request comes in (because we provided it as an argument to the <code>http.createServer()</code> method)
*There are two arguments
**<code>request</code> is an object which represents the request that was received
**<code>response</code> is an object which represents the response that will be sent
*At a high level, this function checks for the existence of a requested file and responds with either the file or an error code
*There are many nested callback functions here
**Although this makes the code more difficult to read, why is this preferable to doing things synchronously?
***This ensures that the <code>request_handler()</code> function will return very quickly
***All the bulk of the work is done asynchronously
***This allows the server to be available to handle another request again very quickly
***This is important in order to keep the server responsive for numerous requests

====MIME Types====
*MIME types are used to specify the type of data being sent via HTTP
*In Tiny Web Server, there is an object <code>MIME_TYPES</code> defined near the top of the file which contains the types of files that the server recognizes
*The <code>MIME_TYPES</code> object is used in the <code>get_mime()</code> function to determine the type of a requested file
*Similarly, the <code>MIME_TYPES</code> object is used to set the Content-Type header of the HTTP response that is sent by the server
*We can break ‘image/jpg’ by changing it to ‘imaage/jpg’
**The web browser does not know how to handle it because it does not recognize imaage
*Web browsers can deal with broken content type for some types of files
**For example, putting a jpg as png in <code>MIME_TYPES</code>, will not break the content and Firefox is smart enough to render it correctly as jpg
**To prevent the browser from attempting to detect what format the content is, the <tt>X-Content-Type-Options: nosniff</tt> header can be added.

====Response Headers====
*Some headers of the HTTP response are set in the <code>respond()</code> function
**This is done using the <code>response.writeHead()</code> method
**We could add more headers here if we want to
*Some other headers are set for us by the code in the <code>http</code> module
*We can see these headers from within the dev tools of the web browser after a response has been received
*There are not many headers used here because Tiny Web Server is not very complex

====Frameworks and Modern Servers====
How have things improved since 1995?
*We use code to generate responses to requests
**We often use frameworks to do much of the work for us

Why use frameworks?
*Because we don’t want to build up HTML from scratch
*Abstracted away logic
*We also want routing (dispatcher): have someone else decide which function to call in response to requests

====Debugging====
*You can use the node debugger by typing <code>node debug script.js</code> in the terminal
*This will stop the program on the first line of code that is run
*You can continue execution of the code by typing <code>c</code>
*You can learn about other useful commands by typing <code>help</code>
*In particular, you can type <code>repl</code> to enter a read-evaluate-print loop from which you can inspect or modify objects or other data
*You can also write <code>debugger;</code> in the script to create a break point where the debugger will pause the execution of the program
*There is also a debugger available in the web browser
**This is for client side code and will not work for any server-side code

====HTML====
*You can use the inspect in the web browser dev tools to see the the tree-structure HTML which makes up a web page
*<code><div></code> tags are used for sections in HTML
*<code><a></code> tags are used for links (review on CodeAcademy)

WebFund 2016W: Tutorial 2

2016-01-20T22:05:01Z

Zero: Quite a few people in the tutorial are getting into trouble with this.

In this tutorial you'll be playing with [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/code/tinywebserver.js tinywebserver.js], a static file webserver in node.js that has no dependencies outside of the core node.js modules. It is based on a [https://github.com/rodw/tiny-node.js-webserver tiny node.js web server] written by [http://heyrod.com/ Rod Waldhoff]. (Note that the original is written in [http://coffeescript.org/ CoffeeScript].)

To run it, just type "node tinywebserver.js". It will begin serving on the web the pages in the same directory as the script. You'll want to have some files files to serve; to start off, try having it serve [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/code/hello.html hello.html] from class. Then, move on to saving pages from the web and serving those. (Note: they probably will be broken in some way by tinywebserver.)

You can run tinywebserver under the debugger by typing "node debug tinywebserver.js". See below for instructions on how to use the node debugger. You may also want to use the browser developer tools to see how the web browser is interacting with the server.

===Questions===
# What port is the web server listening on?
# What file is returned when you access a directory? Why?
# What sort of URL do you give to get the insecure URL warning (403 Forbidden response code)?
# What URL do you need to get a 500 error?

===Tasks===
# Change the port the server listens on to be 3000.
# Change the server to serve files from /home/student/public_html rather than the current directory. Be sure to put some files in it! (Hint: you can grab HTML files from any webserver using wget or curl.)
# Change tinywebserver.js so it returns a simple HTML error page in response to 404 errors (page not found). ''(Hint: Make sure you keep all your JavaScript and HTML files in the same folder!)''
# Make the options object be stored in a file options.json and read on server startup. This load may be synchronous! Use [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse JSON.parse()] to convert the file data into an object.

==Node debugger==

Node has a [http://nodejs.org/api/debugger.html built-in debugger]. Start it by running <tt>node debug tinywebserver.js</tt>. This will stop on the first line of the file. Type <tt>n</tt> to step to the next line of the file. Type <tt>c</tt> to continue to the next breakpoint. Breakpoints are set by adding a <tt>debugger;</tt> statement to the javascript source.

At any time you can type <tt>repl</tt> into the debugger to drop into a read-eval-print loop where you can evaluate JavaScript statements in the current context. Ctrl-C will get you out of the REPL.

For example, consider this source for <tt>test.js</tt>.
<source line lang="javascript">
var x = 5;
var y = 10;

debugger;
</source>
You can run <tt>node debug test.js</tt> to start the debugger.

It which will stop on the first line of the file (<tt>var x = 5;</tt>). If you enter <tt>c</tt> node will continue executing until the <tt>debugger;</tt> statement where it will stop. From here if you enter <tt>repl</tt> you can execute Javascript in the current context. In the <tt>repl</tt> prompt if you enter <tt>x;</tt> it will return 5. If you enter <tt>x + y;</tt> it will return 15, etc.

==Browser developer tools==

* [https://developer.mozilla.org/en-US/docs/Tools Firefox developer tools]: Tools->Web Developer->Toggle Tools
* [https://developer.chrome.com/devtools/index Chrome/Chromium developer tools]: Tools->Developer Tools
The above links have full documentation on the respective developer tools.

Select the Network tab to see HTTP traffic.
Select Inspector (Firefox) or Elements (Chrome/Chromium) to see the HTML document (DOM).

WebFund 2016W Lecture 4

2016-01-19T20:52:37Z

Zero: Added in-class notes.

==Notes==

===In-class Notes===

<pre>
Lecture 4
---------
Web servers

What does a web server do?
Listens on port 80, 443, or other port
INPUT: HTTP requests
OUTPUT: HTTP responses & requested documents

tinywebserver.js
- written in JavaScript (Node)
- can only serve files
- understands only a few file types

So tinywebserver is how web servers worked in 1995

What changed since then is now web servers run code to generate responses

Why use frameworks?
- because we don't want to build up HTML from scratch
- and, we want logic that is abstracted away
- and, we want routing
- have someone else decide which function to call in response to requests

But for this week, bare bones

Debugging

</pre>

WebFund 2016W Lecture 2

2016-01-18T18:34:01Z

Zero: New section added.

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).
* The Mozilla Developer Network maintains a [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference JavaScript reference] that is recommend. ''(Note: There is no official documentation for Google Chrome's V8 engine, although there is an [http://v8.paulfryzel.com/docs/master/index.html unofficial reference].)''

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Synchronous vs. Asynchronous====

* Synchronous (blocking) functions are where a task must be fully complete before the next task can be begin.
** Examples:
*** Waiting on a single network connection before accepting additions requests.
*** Waiting for a file to be fully loaded before displaying it in a text editor.
* Asynchronous (non-blocking) functions are where the program is allowed to continue with other tasks while another one is in process.
** Examples:
*** Accepting new network connections while another connection is already in progress.

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons). These functions have their own set of [https://nodejs.org/api/ documentation].
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Why use Node.js?=====

* JavaScript makes it easier to write asynchronous functions.
* You can reuse your knowledge on client-side JavaScript to apply to server-side Node.js applications.

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]].

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-18T18:22:30Z

Zero: /* Student Notes */

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).
* The Mozilla Developer Network maintains a [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference JavaScript reference] that is recommend. ''(Note: There is no official documentation for Google Chrome's V8 engine, although there is an [http://v8.paulfryzel.com/docs/master/index.html unofficial reference].)''

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons). These functions have their own set of [https://nodejs.org/api/ documentation].
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Why use Node.js?=====

* JavaScript makes it easier to write asynchronous functions.
* You can reuse your knowledge on client-side JavaScript to apply to server-side Node.js applications.

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]].

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-18T18:20:52Z

Zero: /* Node.js */

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).
* The Mozilla Developer Network maintains a [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference JavaScript reference] that is recommend. ''(Note: There is no official documentation for Google Chrome's V8 engine, although there is an [http://v8.paulfryzel.com/docs/master/index.html unofficial reference].)''

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons). These functions have their own set of [https://nodejs.org/api/ documentation].
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Why use Node.js?=====

* JavaScript makes it easier to write asynchronous functions.
* You can reuse your knowledge on client-side JavaScript to apply to server-side Node.js applications.

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-18T18:15:50Z

Zero: /* How should I learn JavaScript? */

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).
* The Mozilla Developer Network maintains a [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference JavaScript reference] that is recommend. ''(Note: There is no official documentation for Google Chrome's V8 engine, although there is an [http://v8.paulfryzel.com/docs/master/index.html unofficial reference].)''

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons). These functions have their own set of [https://nodejs.org/api/ documentation].
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-18T18:10:25Z

Zero: /* Node.js */

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons). These functions have their own set of [https://nodejs.org/api/ documentation].
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-18T18:07:56Z

Zero: /* Working with Variables */

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object (which can optionally encapsulate other objects).
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons).
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W: Assignment 1

2016-01-18T03:00:17Z

Zero: Corrected typo.

This assignment is due on January 20th on cuLearn.

In this assignment you will writing two programs, <tt>words-sync.js</tt> and <tt>words-async.js</tt>. These programs will make use of synchronous and asynchronous I/O is the manner of the programs from [[WebFund 2016W: Tutorial 1|Tutorial 1]].

These programs should both behave as follows:

* They should take two arguments, an input text file and an output file to create. Thus you run them by typing "node words-sync.js document.txt wordlist.txt", with document.txt being the file to be read and wordlist.txt is the file to be created.
* The output file should contain a list of words in the input file, one word per line, with each line terminated by a newline character ('\n'). These words should be sorted in ascending order, as ordered by the standard JavaScript [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort Array.sort() method] (with no comparison function specified). All duplicates should be removed and there should be no blank lines.
* Once the output file has been created, the program should output "Finished!" to the console.
* For the purposes of this program, words can be defined as anything matching the \w operator. In other words, to split the document into words you may call [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split split()] using /\W/ as the split [https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions regular expression].
* You may assume that the input file exists and is readable and the output file is writable. In other words you do not need to do any error checking (although it is appreciated).

In creating this program, don't forget that you can use standard JavaScript methods such as [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/push push()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/pop pop()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/shift shift()], and [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/unshift unshift()]. And of course you'll need the methods from [https://nodejs.org/api/fs.html the Node fs module].

There are 10 points, allocated as follows:
* 3 points for producing the correct output given well-formed input.
* 2 points for correctly structuring the code in the synchronous version.
* 5 points for correctly structuring the code in the asynchronous version. In particular, "Finished!" should only be output when the program is truly finished!

Please submit your answers as a zip file called "<username>-comp2406-assign1.zip", where username is your MyCarletonOne username. This zip file should uncompress to a directory called "<username>-comp2406-assign1" and inside this directory should be three files: "words-sync.js", "words-async.js", and a text file "comments.txt".

"comments.txt" should:
* list any references you used to complete the assignment (documentation web sites, for example),
* list your collaborators, and
* optionally, should discuss any issues or concerns you had when completing this assignment.
Remember that while you are allowed to collaborate with others, plagiarism is not allowed. In other words you should not be copying any code or data directly from anywhere, and any assistance or inspiration should be credited. Any significant code similarity (beyond the code already given to you) will be considered plagiarism and will be reported to the Dean.

WebFund 2016W: Assignment 1

2016-01-18T01:21:09Z

Zero: Undo revision 20534 by Zero (talk)

This assignment is due on January 20th on cuLearn.

In this assignment you will writing two programs, <tt>words-sync.js</tt> and <tt>words-async.js</tt>. These programs will make use of synchronous and asynchronous I/O is the manner of the programs from [[WebFund 2016W: Tutorial 1|Tutorial 1]].

These programs should both behave as follows:

* They should take two arguments, an input text file and an output file to create. Thus you run them by typing "node words-sync.js document.txt wordlist.txt", with document.txt being the file to be read and wordlist.txt is the file to be created.
* The output file should contain a list of words in the input file, one word per line, with each line terminated by a newline character ('\n'). These words should be sorted in ascending order, as ordered by the standard JavaScript [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort Array.sort() method] (with no comparison function specified). All duplicates should be removed and there should be no blank lines.
* Once the output file has been created, the program should output "Finished!" to the console.
* For the purposes of this program, words can be defined as anything matching the \w operator. In other words, to split the document into words you may call [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split split()] using "/\W//" as the split [https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions regular expression].
* You may assume that the input file exists and is readable and the output file is writable. In other words you do not need to do any error checking (although it is appreciated).

In creating this program, don't forget that you can use standard JavaScript methods such as [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/push push()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/pop pop()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/shift shift()], and [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/unshift unshift()]. And of course you'll need the methods from [https://nodejs.org/api/fs.html the Node fs module].

There are 10 points, allocated as follows:
* 3 points for producing the correct output given well-formed input.
* 2 points for correctly structuring the code in the synchronous version.
* 5 points for correctly structuring the code in the asynchronous version. In particular, "Finished!" should only be output when the program is truly finished!

Please submit your answers as a zip file called "<username>-comp2406-assign1.zip", where username is your MyCarletonOne username. This zip file should uncompress to a directory called "<username>-comp2406-assign1" and inside this directory should be three files: "words-sync.js", "words-async.js", and a text file "comments.txt".

"comments.txt" should:
* list any references you used to complete the assignment (documentation web sites, for example),
* list your collaborators, and
* optionally, should discuss any issues or concerns you had when completing this assignment.
Remember that while you are allowed to collaborate with others, plagiarism is not allowed. In other words you should not be copying any code or data directly from anywhere, and any assistance or inspiration should be credited. Any significant code similarity (beyond the code already given to you) will be considered plagiarism and will be reported to the Dean.

WebFund 2016W: Assignment 1

2016-01-17T04:29:23Z

Zero: There's no darn quotes. That's half an hour I'm never going to get back.

This assignment is due on January 20th on cuLearn.

In this assignment you will writing two programs, <tt>words-sync.js</tt> and <tt>words-async.js</tt>. These programs will make use of synchronous and asynchronous I/O is the manner of the programs from [[WebFund 2016W: Tutorial 1|Tutorial 1]].

These programs should both behave as follows:

* They should take two arguments, an input text file and an output file to create. Thus you run them by typing "node words-sync.js document.txt wordlist.txt", with document.txt being the file to be read and wordlist.txt is the file to be created.
* The output file should contain a list of words in the input file, one word per line, with each line terminated by a newline character ('\n'). These words should be sorted in ascending order, as ordered by the standard JavaScript [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort Array.sort() method] (with no comparison function specified). All duplicates should be removed and there should be no blank lines.
* Once the output file has been created, the program should output "Finished!" to the console.
* For the purposes of this program, words can be defined as anything matching the \w operator. In other words, to split the document into words you may call [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split split()] using /\W// as the split [https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions regular expression].
* You may assume that the input file exists and is readable and the output file is writable. In other words you do not need to do any error checking (although it is appreciated).

In creating this program, don't forget that you can use standard JavaScript methods such as [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/push push()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/pop pop()], [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/shift shift()], and [https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/unshift unshift()]. And of course you'll need the methods from [https://nodejs.org/api/fs.html the Node fs module].

There are 10 points, allocated as follows:
* 3 points for producing the correct output given well-formed input.
* 2 points for correctly structuring the code in the synchronous version.
* 5 points for correctly structuring the code in the asynchronous version. In particular, "Finished!" should only be output when the program is truly finished!

Please submit your answers as a zip file called "<username>-comp2406-assign1.zip", where username is your MyCarletonOne username. This zip file should uncompress to a directory called "<username>-comp2406-assign1" and inside this directory should be three files: "words-sync.js", "words-async.js", and a text file "comments.txt".

"comments.txt" should:
* list any references you used to complete the assignment (documentation web sites, for example),
* list your collaborators, and
* optionally, should discuss any issues or concerns you had when completing this assignment.
Remember that while you are allowed to collaborate with others, plagiarism is not allowed. In other words you should not be copying any code or data directly from anywhere, and any assistance or inspiration should be credited. Any significant code similarity (beyond the code already given to you) will be considered plagiarism and will be reported to the Dean.

Fundamentals of Web Applications (Winter 2016)

2016-01-17T01:00:02Z

Zero: Typo.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class)

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|TBA]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 27

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 3

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 10|Assignment 10]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity. Keep in mind that the browser is likely out of date and you should not log into any important services (e.g., Gmail) due to missing security patches.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

====Working directly off network drive (sshfs)====

If you don't want to bother running save2406 and restore2406 all the time, you can optionally mount the network drive.

sudo apt-get install sshfs # You may already have this package
mkdir ~/network-drive/ # Pick any name you want.
sshfs -o reconnect,transform_symlinks ~/network-drive/ <SCS username>@access.scs.carleton.ca:/

At this point, you should see all of your files in ~/network-drive/COMP2406/.

Note: There is no concept of "restore" or undo. If you delete or overwrite a file, it's gone.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova virtual appliance file] and import into [http://www.virtualbox.org VirtualBox] or VMware Workstation/Fusion ([https://secure.scs.carleton.ca/vmware/ free to Carleton SCS students]). The SHA1 hash of this file is:

97fc4de4544505178b580d27487085a20961e91e [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova COMP 2406, Winter 2016.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If your virtualization platform is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in native Windows. Here are some tips on getting things running.

* You can install the [https://nodejs.org/en/download/ official Node.js package] to get basic node and npm functionality.
* See [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].
* [https://atom.io/ Atom] is a another popular free GUI text editor.

===Configuring Mac OS X===

It is possible, and recommended, to configure Node and mongoDB to run on your Mac.

====Homebrew method (recommended)====
Homebrew is a package manager for OS X that works similarly to Linux-style package managers like <tt>apt</tt>. Homebrew provides access to thousands of Linux and BSD packages to install natively on your Mac. You have a perfectly capable UNIX system in front of you. Make good use of it!

* Install [http://brew.sh/ Homebrew] (copy the installation text on that page to Terminal.app)
* Install Node and mongoDB

Run the following commands in your terminal:

brew update
brew install node
brew install mongo

That's it!

====Manual method====
Alternatively, you can download and install Node and mongo directly.

* Download and install the ''Mac OS X Installer'' from [https://nodejs.org/en/download/stable/ the download page].
* Download the binaries from [https://www.mongodb.org/downloads#production MongoDB's download page]
* Follow [https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-manually these instructions]

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

Lee or Anil will set you up with an account on this wiki. You'll enter your initial draft notes here and then work with Lee to make sure they are of sufficient quality. This may require a few rounds of revisions; however, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

Fundamentals of Web Applications (Winter 2016)

2016-01-17T00:59:44Z

Zero: Added sshfs.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class)

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|TBA]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 27

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 3

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 10|Assignment 10]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity. Keep in mind that the browser is likely out of date and you should not log into any important services (e.g., Gmail) due to missing security patches.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

====Working directly off network drive (sshfs)====

If you don't want to bother running save2406 and restore2406 all the time, you can optionally mount the network drive.

sudo apt-get install sshfs # You may already have this package
mkdir ~/network-drive/ # Pick any name you want.
sshfs -o reconnect,transform_symlinks ~/network-drive/ <SCS username@access.scs.carleton.ca:/

At this point, you should see all of your files in ~/network-drive/COMP2406/.

Note: There is no concept of "restore" or undo. If you delete or overwrite a file, it's gone.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova virtual appliance file] and import into [http://www.virtualbox.org VirtualBox] or VMware Workstation/Fusion ([https://secure.scs.carleton.ca/vmware/ free to Carleton SCS students]). The SHA1 hash of this file is:

97fc4de4544505178b580d27487085a20961e91e [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova COMP 2406, Winter 2016.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If your virtualization platform is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in native Windows. Here are some tips on getting things running.

* You can install the [https://nodejs.org/en/download/ official Node.js package] to get basic node and npm functionality.
* See [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].
* [https://atom.io/ Atom] is a another popular free GUI text editor.

===Configuring Mac OS X===

It is possible, and recommended, to configure Node and mongoDB to run on your Mac.

====Homebrew method (recommended)====
Homebrew is a package manager for OS X that works similarly to Linux-style package managers like <tt>apt</tt>. Homebrew provides access to thousands of Linux and BSD packages to install natively on your Mac. You have a perfectly capable UNIX system in front of you. Make good use of it!

* Install [http://brew.sh/ Homebrew] (copy the installation text on that page to Terminal.app)
* Install Node and mongoDB

Run the following commands in your terminal:

brew update
brew install node
brew install mongo

That's it!

====Manual method====
Alternatively, you can download and install Node and mongo directly.

* Download and install the ''Mac OS X Installer'' from [https://nodejs.org/en/download/stable/ the download page].
* Download the binaries from [https://www.mongodb.org/downloads#production MongoDB's download page]
* Follow [https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-manually these instructions]

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

Lee or Anil will set you up with an account on this wiki. You'll enter your initial draft notes here and then work with Lee to make sure they are of sufficient quality. This may require a few rounds of revisions; however, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

Fundamentals of Web Applications (Winter 2016)

2016-01-17T00:51:11Z

Zero: Security warning.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class)

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|TBA]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 27

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 3

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 10|Assignment 10]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity. Keep in mind that the browser is likely out of date and you should not log into any important services (e.g., Gmail) due to missing security patches.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova virtual appliance file] and import into [http://www.virtualbox.org VirtualBox] or VMware Workstation/Fusion ([https://secure.scs.carleton.ca/vmware/ free to Carleton SCS students]). The SHA1 hash of this file is:

97fc4de4544505178b580d27487085a20961e91e [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova COMP 2406, Winter 2016.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If your virtualization platform is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in native Windows. Here are some tips on getting things running.

* You can install the [https://nodejs.org/en/download/ official Node.js package] to get basic node and npm functionality.
* See [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].
* [https://atom.io/ Atom] is a another popular free GUI text editor.

===Configuring Mac OS X===

It is possible, and recommended, to configure Node and mongoDB to run on your Mac.

====Homebrew method (recommended)====
Homebrew is a package manager for OS X that works similarly to Linux-style package managers like <tt>apt</tt>. Homebrew provides access to thousands of Linux and BSD packages to install natively on your Mac. You have a perfectly capable UNIX system in front of you. Make good use of it!

* Install [http://brew.sh/ Homebrew] (copy the installation text on that page to Terminal.app)
* Install Node and mongoDB

Run the following commands in your terminal:

brew update
brew install node
brew install mongo

That's it!

====Manual method====
Alternatively, you can download and install Node and mongo directly.

* Download and install the ''Mac OS X Installer'' from [https://nodejs.org/en/download/stable/ the download page].
* Download the binaries from [https://www.mongodb.org/downloads#production MongoDB's download page]
* Follow [https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-manually these instructions]

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

Lee or Anil will set you up with an account on this wiki. You'll enter your initial draft notes here and then work with Lee to make sure they are of sufficient quality. This may require a few rounds of revisions; however, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

Fundamentals of Web Applications (Winter 2016)

2016-01-17T00:45:59Z

Zero: Minor bits of info for the M$ users.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class)

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|TBA]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 27

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 3

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 10|Assignment 10]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova virtual appliance file] and import into [http://www.virtualbox.org VirtualBox] or VMware Workstation/Fusion ([https://secure.scs.carleton.ca/vmware/ free to Carleton SCS students]). The SHA1 hash of this file is:

97fc4de4544505178b580d27487085a20961e91e [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406,%20Winter%202016.ova COMP 2406, Winter 2016.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If your virtualization platform is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in native Windows. Here are some tips on getting things running.

* You can install the [https://nodejs.org/en/download/ official Node.js package] to get basic node and npm functionality.
* See [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].
* [https://atom.io/ Atom] is a another popular free GUI text editor.

===Configuring Mac OS X===

It is possible, and recommended, to configure Node and mongoDB to run on your Mac.

====Homebrew method (recommended)====
Homebrew is a package manager for OS X that works similarly to Linux-style package managers like <tt>apt</tt>. Homebrew provides access to thousands of Linux and BSD packages to install natively on your Mac. You have a perfectly capable UNIX system in front of you. Make good use of it!

* Install [http://brew.sh/ Homebrew] (copy the installation text on that page to Terminal.app)
* Install Node and mongoDB

Run the following commands in your terminal:

brew update
brew install node
brew install mongo

That's it!

====Manual method====
Alternatively, you can download and install Node and mongo directly.

* Download and install the ''Mac OS X Installer'' from [https://nodejs.org/en/download/stable/ the download page].
* Download the binaries from [https://www.mongodb.org/downloads#production MongoDB's download page]
* Follow [https://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-manually these instructions]

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

Lee or Anil will set you up with an account on this wiki. You'll enter your initial draft notes here and then work with Lee to make sure they are of sufficient quality. This may require a few rounds of revisions; however, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

WebFund 2016W Lecture 2

2016-01-15T03:16:50Z

Zero: Typo. Thanks Lee.

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object.
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slight differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons).
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 3

2016-01-14T20:53:55Z

Zero: Added in-class notes.

==Notes==

===In-class Notes===

<pre>
Lecture 3
---------
What is the web?

Web != Internet

So, what is the Internet?!

Network of networks

What is a network?

Networks allow computers to talk to each other

WiFi, Ethernet, LTE - different standards for computer networks

Layers of networking (OSI)
* Physical layer
* Data link layer
* Network layer <----
* Application layer (and such)

Internet Protocol (IP)
- packet-based protocol (NOT a stream or continuous protocol)

Old POTS
- wire from your house to a switching station
- wires between switching stations A and B
...
...
- wires between switching station Y to Z
- wire from switching station to house

Packet switching networks multiplex physical wires over time. (Multiple communication connections over one wire by them taking turns.)

The "turn" is the packet

IP protocol is a "best effort" protocol. NO error correction or retransmission.

TCP means Transmission Control Protocol
turns packets into a reliable data data stream

Network Firewalls block "unwanted traffic"
- really, block all those protocols that aren't safe on the open Internet

TCP adds a "port" to the IP address
- the port identifies which program to talk to
- some are ephemeral (temporary)
- others are "well known", meaning protocol on that port is standardized

Port 25 is for SMTP (email)
80: HTTP
443: HTTPS
22: ssh

Web is transmitted over HTTP

Major HTTP commands
* GET: get documents from server
- can be CACHED
* POST: send form contents to server
- are not cached

You can GET almost anything
- MS Word .doc
- tiff
- PDF

But the real web is
- JPEG, GIF, PNG for images
- CSS for style sheets
- HTML for content
- JavaScript for code

.swf is flash. Flash is not a web standard. AVOID

</pre>

WebFund 2016W Lecture 2

2016-01-14T15:50:44Z

Zero: Added bit about garbage collectors.

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object.
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).
* JavaScript engines use a garbage collector, so you do not need to manually deallocate memory (in C you need to). Unlike Java, you can "reuse" a variable name and use a completely different type; e.g. creating <tt>var x = 1337</tt> and then <tt>x = "Dave"</tt> later is completely valid.

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slightly differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons).
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-14T15:46:03Z

Zero: Typo fix.

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contributing to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object.
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slightly differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons).
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-14T15:44:55Z

Zero: Added section on Node.js.

==Video==

The video from the lecture given on January 12, 2016 [http://homeostasis.scs.carleton.ca/~soma/webfund-2016w/lectures/comp2406-2016w-lec02-12Jan2016.mp4 is now available].

==Notes==

===In-class Notes===

<pre>
Lecture #2
----------
JavaScript

* no declared types
- strings
- objects
- arrays
- numbers (floats)
- boolean

With Node
- asynchronous (non-blocking) I/O
</pre>

===Student Notes===

* Administration Reminders
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Course_Information|Office hours have been added.]]
** [[Fundamentals_of_Web_Applications:_Winter_2016_Course_Outline#Grading|Midterm date has been tentatively created.]]
** [[WebFund_2016W:_Assignment_1|Assignment #1 has been posted.]]

====JavaScript: As a Language====

* JavaScript has '''nothing''' to do with Java.
** Why is it even called JavaScript then? At first it wasn't; it was developed as Mocha, changed to LiveScript and then finally renamed to JavaScript due to an agreement with Sun (acquired by Oracle).<ref>[http://stackoverflow.com/a/2475528]</ref>
* The only similarity between Java/JavaScript is some of the syntax, but that can be said for a lot of languages.

* What's different about JavaScript?
** No declared data types; it's a "loose" language.
*** Instead of int i = 0, it's simply var i = 0.
** There's seven data types<ref>[https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures]</ref> (examples in brackets)
*** Boolean (true, false)
*** Number (3.1337, 1337, etc. There's no integer or long, everything is a float)
*** String ("Dave")
*** Symbol
*** Object
*** Null
*** Undefined

====How should I learn JavaScript?====

* Practice! While attending the lectures is a great start, you ''will'' have to practice outside of class. This will not be graded, but will ultimately affect your performance in the course (and in the workplace).
** [https://www.codecademy.com/learn/javascript Codecademy] has a lot of good resources for beginners.
** For more advanced students, check on [https://github.com/search?l=javascript&o=desc&q=stars%3A%3E1&s=stars&type=Repositories GitHub] to see if there's any projects you're interested in contribution to. If you plan to work in web development, it's very likely you will be asked in an interview if you've contributed to any open source projects (having a good portfolio will help a ''lot'' in getting a first job).

====Working with Variables====

If you've previously worked with Java, the first large difference you've noticed is likely that variables are very "flexible".

* If you attempt to use a variable that has not been declared, you will '''not''' get a compiler error. Instead, the type <tt>undefined</tt> will be returned. This can making debugging a bit of a challenge!
* Likewise with most languages, a symbol (e.g., <tt>x</tt>) can only reference ''one'' object.
* You can have several types of data inside single object (e.g., <tt>x.y</tt>, <tt>x.y.z</tt>, <tt>x.y.x.qwerty</tt>, etc).
* <tt>x.size</tt> (dot notation) is the same as <tt>x["size"]</tt> (array notation).

====Regular Expressions (Regex)====

While this topic is technically outside of the course material, you will likely need to use it in some assignments. Regex is the same across different languages, so you will not need to relearn it all for every language.

https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions

====Node.js====

Node.js is designed to provide a server side environment of JavaScript. Instead of using Google's V8 JavaScript inside Chrome/Chromium, Node.js uses the V8 engine directly. Because Node.js is meant to be used outside of the browser, there's some slightly differences.

* Contains a package manager called npm to manage dependencies (libraries).
* Has specialized libraries for networking, file system access, etc that you usually wouldn't find in a browser (mostly for security reasons).
* Has no default graphic frontend such as WebKit (although there are libraries to provide that if you wish).

=====Getting started with Node.js=====

To open up Read-Eval-Print-Loop (REPL), run <tt>node</tt> in a terminal. For more details on setting your environment up, see [[WebFund_2016W:_Tutorial_1#Running_the_VM|Tutorial 1]]

==References==
{{Reflist}}

WebFund 2016W Lecture 2

2016-01-14T15:31:24Z

Zero: Added link.

Fundamentals of Web Applications (Winter 2016)

2016-01-13T20:54:57Z

Zero: Added poll link.

==Course Outline==

[[Fundamentals of Web Applications: Winter 2016 Course Outline|Here]] is the course outline.

==Lectures and Exams==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 7

</td>
<td>
[[WebFund 2016W Lecture 1|Lecture 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[WebFund 2016W Lecture 2|Lecture 2]]

</td>
</tr>
<tr>
<td>
Jan. 14

</td>
<td>
[[WebFund 2016W Lecture 3|Lecture 3]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[WebFund 2016W Lecture 4|Lecture 4]]

</td>
</tr>
<tr>
<td>
Jan. 21

</td>
<td>
[[WebFund 2016W Lecture 5|Lecture 5]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[WebFund 2016W Lecture 6|Lecture 6]]

</td>
</tr>
<tr>
<td>
Jan. 28

</td>
<td>
[[WebFund 2016W Lecture 7|Lecture 7]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[WebFund 2016W Lecture 8|Lecture 8]]

</td>
</tr>
<tr>
<td>
Feb. 4

</td>
<td>
[[WebFund 2016W Lecture 9|Lecture 9]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[WebFund 2016W Lecture 10|Midterm Review]]

</td>
</tr>
<tr>
<td>
Feb. 11

</td>
<td>
Midterm (in class) (Tentative, [https://culearn.carleton.ca/moodle/mod/choice/view.php?id=779191 click here for poll.])

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[WebFund 2016W Lecture 11|Lecture 11]]

</td>
</tr>
<tr>
<td>
Feb. 25

</td>
<td>
[[WebFund 2016W Lecture 12|Lecture 12]]

</td>
</tr>
<tr>
<td>
Mar. 1

</td>
<td>
[[WebFund 2016W Lecture 13|Lecture 13]]

</td>
</tr>
<tr>
<td>
Mar. 3

</td>
<td>
[[WebFund 2016W Lecture 14|Lecture 14]]

</td>
</tr>
<tr>
<td>
Mar. 8

</td>
<td>
[[WebFund 2016W Lecture 15|Lecture 15]]

</td>
</tr>
<tr>
<td>
Mar. 10

</td>
<td>
[[WebFund 2016W Lecture 16|Lecture 16]]

</td>
</tr>
<tr>
<td>
Mar. 15

</td>
<td>
[[WebFund 2016W Lecture 17|Lecture 17]]

</td>
</tr>
<tr>
<td>
Mar. 17

</td>
<td>
[[WebFund 2016W Lecture 18|Lecture 18]]

</td>
</tr>
<tr>
<td>
Mar. 22

</td>
<td>
[[WebFund 2016W Lecture 19|Lecture 19]]

</td>
</tr>
<tr>
<td>
Mar. 24

</td>
<td>
[[WebFund 2016W Lecture 20|Lecture 20]]

</td>
</tr>
<tr>
<td>
Mar. 29

</td>
<td>
[[WebFund 2016W Lecture 21|Lecture 21]]

</td>
</tr>
<tr>
<td>
Mar. 31

</td>
<td>
[[WebFund 2016W Lecture 22|Lecture 22]]

</td>
</tr>
<tr>
<td>
Apr. 5

</td>
<td>
[[WebFund 2016W Lecture 23|Lecture 23]]

</td>
</tr>
<tr>
<td>
April 7

</td>
<td>
[[WebFund 2016W Lecture 24|Lecture 24]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
Final Exam

</td>
</tr>
</table>

==Tutorials==

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Date
</th>
<th>
Tutorials
</th>
</tr>
<tr>
<td>
Jan. 13, 14, 15, 18

</td>
<td>
[[WebFund 2016W: Tutorial 1|Asynchronous Code in JavaScript]]

</td>
</tr>
<tr>
<td>
Jan. 20, 21, 22, 25

</td>
<td>
[[WebFund 2016W: Tutorial 2|Tiny web server]]

</td>
</tr>
<tr>
<td>
Jan. 27, 28, 29, Feb. 1

</td>
<td>
[[WebFund 2016W: Tutorial 3|Express, Form demo]]

</td>
</tr>
<tr>
<td>
Feb. 3, 4, 5, 8

</td>
<td>
[[WebFund 2016W: Tutorial 4|TBA]]

</td>
</tr>
<tr>
<td>
Feb. 24, 25, 26, 29

</td>
<td>
[[WebFund 2016W: Tutorial 5|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 2, 3, 4, 7

</td>
<td>
[[WebFund 2016W: Tutorial 6|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 9, 10, 11, 14

</td>
<td>
[[WebFund 2016W: Tutorial 7|TBA]]

</td>
</tr>
<tr>
<tr>
<td>
Mar. 16, 17, 18, 21

</td>
<td>
[[WebFund 2016W: Tutorial 8|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 23, 24, 25, 28

</td>
<td>
[[WebFund 2016W: Tutorial 9|TBA]]

</td>
</tr>
<tr>
<td>
Mar. 30, 31, Apr. 1, 4

</td>
<td>
[[WebFund 2016W: Tutorial 10|TBA]]

</td>
</tr>
</table>

==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<tr valign="top">
<th>
Due Date
</th>
<th>
Assignments
</th>
</tr>
<tr>
<td>
Jan. 20

</td>
<td>
[[WebFund 2016W: Assignment 1|Assignment 1]]

</td>
</tr>
<tr>
<td>
Jan. 27

</td>
<td>
[[WebFund 2016W: Assignment 2|Assignment 2]]

</td>
</tr>
<tr>
<td>
Feb. 3

</td>
<td>
[[WebFund 2016W: Assignment 3|Assignment 3]]

</td>
</tr>
<tr>
<td>
Feb. 9, 2:30 PM

</td>
<td>
[[WebFund 2016W: Assignment 4|Assignment 4]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[WebFund 2016W: Assignment 5|Assignment 5]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[WebFund 2016W: Assignment 6|Assignment 6]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[WebFund 2016W: Assignment 7|Assignment 7]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[WebFund 2016W: Assignment 8|Assignment 8]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[WebFund 2016W: Assignment 9|Assignment 9]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[WebFund 2016W: Assignment 10|Assignment 10]]

</td>
</tr>
</table>

==Course Software==

In this course we will be developing web applications using [http://nodejs.org/ node.js] and [http://www.mongodb.org/ mongoDB]. You are welcome to use whatever operating system and development tools you like; however, we will supporting the use of a course virtual machine appliance running [http://www.lubuntu.net/ Lubuntu], a low-resource variant of [http://www.ubuntu.com/ Ubuntu] Linux distribution.

===In the labs===

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406 virtual machine image. After the VM has fully booted up you can login to the student account using the password "tneduts!". This account has administrative privileges; in addition, there is the admin account in case your student account gets corrupted for any reason. The password for it is "nimda!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

save2406 <SCS username>
restore2406 <SCS username>
compare2406 <SCS username>

If you use these commands, '''use them consistently'''. That means run <tt>restore2406</tt> when you first log in, and run <tt>save2406</tt> just before logging out. If you don't do this, you will '''erase''' the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP2406/

This is the same as the <tt>save2406</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the <tt>-n</tt> option to do a dry run.

===Running the VM on your own machines===

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406%20&%203000,%20Fall%202015.ova virtual appliance file] and import. The SHA1 hash of this file is:

a8a70ec2e1b49699f4de29c872ecec7cee21888f [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406%20&%203000,%20Fall%202015.ova COMP 2406 & 3000, Fall 2015.ova]

On Windows you can compute this hash for your downloaded file using the command <a href="http://support.microsoft.com/kb/889768"><tt>FCIV -sha1 COMP 2406 & 3000, Fall 2015.ova</tt></a>. If the hash is different from above, your download has been corrupted.

If the application is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
sudo /opt/VBoxGuestAdditions-*/uninstall.sh
* Install your platform's own Linux guest additions, if available.

===Configuring Linux===

If you already run Linux and you want to use the same packages we do in class, you should do the following:

* Install the latest node.js version using [https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager a package manager]. (You generally don't use the version that comes with your distribution, it is probably too old.)
* Install [http://www.mongodb.org/ mongoDB] - it probably goes by mongodb in your package manager. The version doesn't matter too much.

Note that the binary of the node.js executable may be <tt>node</tt>, <tt>nodejs</tt>, or something similar.

That's it!

===Configuring Windows===

It is also possible to do everything in this class in Windows. Here are some tips on getting things running.

* You can just install the [http://nodejs.org/download/ windows node package] to get basic node and npm functionality.
* [[Run MongoDB on Windows]].
* If you want to run Emacs on Windows, look at the [[Running Emacs on Windows|following tips]].

==Resources==

In general any text published by [http://www.oreilly.com/ O'Reilly] is worth your time. Many (but not all) of the texts below are published by them.

===JavaScript===

[http://shop.oreilly.com/product/9780596517748.do JavaScript: The Good Parts] by Douglas Crockford is an excellent text on JavaScript and it is short. We highly suggest you obtain a copy and read it.

The easiest way to get started with JavaScript and get basic understanding of web technologies is to go through the interactive lessons on [http://codeacademy.com Code Academy]. I suggest you go through their JavaScript, Web Fundamentals, and jQuery tracks. They shouldn't take you very long to do given that you already know how to program.

Crockford also has a lot of online resources on JavaScript, including videos of talks he's given that cover much of the content in his book. Look at his [http://javascript.crockford.com/ JavaScript page] and this [http://yuiblog.com/crockford/ page of his videos].

Another good book is [http://eloquentjavascript.net/ Eloquent JavaScript: A Modern Introduction to Programming] by Marijn Haverbeke. A version of this book is available online for free. The for-sale version is apparently updated and edited.

The standard reference for JavaScript is [http://shop.oreilly.com/product/9780596805531.do JavaScript: The Definitive Guide] by David Flanagan. It is a big book, but it is comprehensive.

===Node===

[http://shop.oreilly.com/product/0636920024606.do Learning Node] by Shelley Powers is the recommended resource for learning about Node.

Another introduction to node.js is [http://www.nodebeginner.org/ The Node Beginner Book] by Manuel Kiessling.

[http://evanhahn.com/understanding-express-js/ This page by Evan Hahn] has a good overview of the express/connect/node software stack.

===MongoDB===

[http://shop.oreilly.com/product/0636920028031.do MongoDB: The Definitive Guide] by Kristina Chodorow (1st or 2nd ed) is the recommended text for learning more about MongoDB.

==Lecture Notes Guidelines==

As specified in the course outline you can get up to 6% extra credit on your final grade (2% per time) for writing up notes from lecture. Here are the guidelines for those notes.

Lee Croft (LeeCroft at cmail.carleton.ca) will be handling course notes. Please contact him if you would like to volunteer. Note that you should email him in advance and he will give you the next available slot. There are over 200 students in class and only 20 lectures so opportunities may go fast!

Lee or Anil will set you up with an account on this wiki. You'll enter your initial draft notes here and then work with Lee to make sure they are of sufficient quality. This may require a few rounds of revisions; however, if you follow the guidelines below it shouldn't be too bad.

You should plan on organizing your notes as follows:
* Organize them in at least the following sections: Topics & Readings, Audio & Video, and Notes.
* The Topics & Readings section lists the main topics covered in the course, e.g. "Scoping rules in JavaScript". Please use an unordered bulleted list (using *'s in wiki markup). In this section also list readings relevant to the lecture that were mentioned in class.
* Leave the Audio and Video section blank. Anil will fill this out.
* Put your notes in the Notes section.

Use (nested) lists if appropriate for the notes; however, please have some text that isn't bulleted. Please try to make the notes even if you did not attend lecture; however, you don't need to cover every small bit of information that was covered. In particular the notes do not need to include digressions into topics only tangentially related to the course. Complete sentences are welcome but not required.

WebFund 2016W Lecture 2

2016-01-13T20:50:15Z

Zero: /* Student Notes */

WebFund 2016W Lecture 2

2016-01-13T20:43:25Z

Zero: /* Working with Variables */

WebFund 2016W Lecture 2

2016-01-13T20:33:45Z

Zero: Added new section.

WebFund 2016W Lecture 2

2016-01-13T20:29:47Z

Zero: Added new section.

WebFund 2016W Lecture 2

2016-01-13T20:17:56Z

Zero: Broken URL.

WebFund 2016W Lecture 2

2016-01-13T20:16:55Z

Zero: Removed the ref titles.