WebFund 2016W: Assignment 4

In this assignment you will create queryLogs.js, a program that will query the logs collection in the log-demo database for matching log entries. Please submit your answers as a single JavaScript source file called "<username>-queryLogs.js" (where username is your MyCarletonOne username). Please do not submit a zip file (i.e., no need for a node_modules directory). Note that this means your code should not use any modules beyond standard node modules and mongodb.

Please list any collaborators and resources you used as comments in your source file.

In total, there are 10 points. The questions below will be automatically graded, so please follow the directions carefully.

This assignment is due March 3, 2016.

Syntax

queryLogs.js should be called as follows:

 node queryLogs.js [--results=<output file>] [--maxcount=<max number of documents to return>]  [--service="service query"]
     [--message="message query"]

Here is an example of a query that includes all the options:

 node queryLogs.js --results=out.txt --maxcount=10 --service="systemd" --message="[Ss]tarted" 

The above would return at most 10 matching log messages and output them to out.txt. Key points:

• The log lines should be outputted as lines, not as objects.
• The output lines should appear as they were before being stored in the database. (Please do not include the file or ID fields.)
• Be sure not to forget the colon after the service! However, it is okay if multiple spaces are converted to one space in your output. The ordering of the outputted lines is likely to be different, especially as the lines may come from multiple log files that were stored.
• The service and message queries should both be regular expressions and should be combined with a logical and if both are present.
• All arguments are optional. If neither --service or --message are specified, all stored log messages should be returned (up to --maxcount, if specified). If no --maxcount is specified, all matching records should be output. If no --results is specified, the output should go to standard out. Thus, node queryLogs.js can be called with no arguments and should just dump all of the log messages (albeit potentially in an arbitrary order).

In order to query anything you'll need a working version of storeLogs.js from Tutorial 5. be sure to store the logs in the logs collection in the log-demo database, just as storeLogLine.js does.

Scoring

Points will be awarded as follows:

• 2 for --results
• 2 for --maxcount
• 2 for --service
• 2 for --message
• 2 for functionality with missing arguments

Total: 10 points

Note that if your code exits with an error you may get 0 points. Make sure you test your code with log files other than the one supplied (look at /var/log/syslog for more entries!).

Hints

• You'll be using the find() as we did in class with mongodb (see examforms-mongodb that we did in Lecture 12). For the query object, the properties are the fields you want to query and the value is an object that describes the query. MongoDB supports many operators; here you'll want the $regex operator. So for example, to search for any names that start with "Bel", and are in a city with a name ending in "ville" you'd do something like this:

 var customersCollection = db.collection("customers");
 var cursor = customersCollection.find({name: {$regex: "^Bel"}, city: {$regex: "ville$"}});

• You can change how the cursor object acts using many operators. One is the limit method. So if you wanted any find operation on this cursor to only return 5 results, run:

 cursor.limit(5);

• To actually get all the results at once, run the toArray method for cursor objects. See examforms-mongodb for an example.

• The mongodb syntax is different in the version 1 and version 2 of the node driver. I suggest using the version 2 interface as it is cleaner.

• The regular expression /--(\w+)=(.+)/ plus the string match method is very helpful for parsing command line arguments.

Good luck!

Solutions

Download

// queryLogs.js
// solutions to Assignment 4
// Carleton University COMP 2406, 2016W

var fs = require('fs');
var mc = require('mongodb').MongoClient;
var db, logs;

var matches;
var argRE = /--(\w+)=(.+)/;

var options = {
    results: '',
    maxcount: '0',
    service: '',
    message: ''
}

for (i=2; i<process.argv.length; i++) {
    matches = process.argv[i].match(argRE);

    if (matches) {
        options[matches[1]] = matches[2];
    }
}

var reportResults = function(err, results) {
    var lines = [];

    if (err) {
	throw err;
    }

    results.forEach(function(entry) {
	var s = [entry.date, entry.time, entry.host, entry.service + ":",
		 entry.message];
	lines.push(s.join(' '));	
    });

    if (options.results ==='') {
	console.log(lines.join('\n'));
    } else {
	fs.writeFileSync(options.results, lines.join('\n'), 'utf-8');
    }
    db.close();
}

var connectCallback = function(err, returnedDB) {
    if (err) {
	throw err;
    }

    db = returnedDB;
    logs = db.collection('logs');
    logs.find({service: {$regex: options.service},
	       message: {$regex: options.message}}
	     ).limit(parseInt(options.maxcount)).toArray(reportResults);
}

mc.connect('mongodb://localhost/log-demo', connectCallback);