COMP 2406 2024F Assignment 3 Solutions

GRADING NOTE: If a description of the development process or issues encountered is omitted, at least half the points of the question are deducted.


1. [2] How could you change submitdemo so it accepted submissions named comp2406-assign3-soma.txt or similarly for other usernames? Be sure to specify all the places in the code that must be changed. Assume only valid submissions are uploaded.

A: The filename is only checked in validator.js in the original version of submitdemo, and it is specified in the filePrefix constant. Change the value of this constant on line 10 from "comp2406-tutorial7" to "comp2406-assign3" and you're done!

(Note that this question does not ask how to update submitdemo so works completely for assignment 3. You'd also need to change the table name and the acceptable first line of the submission.)

(1 point for identifying filePrefix, 1 point for recognizing nothing else needed to be changed.)


2. [2] If you create a submission that is valid except for having the wrong number of questions (i.e., not five), such a submission will be allowed to be upload even though it is invalid. Why? And what is a quick but correct way to fix this problem?

A: In checkSubmission() in validator.js, errors are mostly indicated by updating the status tag with the error and then returning from the function with no value. When parsing completes successfully at the end of the function, q is returned with the parsed submission which can then be uploaded.

However, in lines 137-140, when there is a check against the expectedQuestionList, the code does not return when there is an error, and in fact parsing continues thus allowing q (in its not fully validated state) to be returned and thus potentially uploaded.

To fix this, we just need to insert a "return" between lines 140 and 141 (the end of the if block). With this change, the function will stop executing when a question number error is encountered and it won't return anything that could be uploaded.

(1 for the explanation, 1 for the fix)


3. [2] How could you change submitdemo so submissions have six questions rather than five? Be sure to specify all the places in the code that must be changed. Assume that only valid submissions are uploaded.

A: To increase the number of questions in a submission, we have to change what is validated and what the server stores in the database.

First, we need to change the expectedQuestionList constant on line 12 of validator.js:

const expectedQuestionList = "1,2,3,4,5,6";

This will allow a six-question submission to be uploaded.

Then we have to make several changes to submitdemo.js. We have to change the number of questions when the database is created (replace line 34 with this):

       q5 TEXT,
       q6 TEXT

We have to change how submissions are added (inserted) in addSubmissionDB(), changing the db.query() to the following (making 3 changes):

    return db.query(`INSERT INTO ${table} ` +
                    "(studentID, name, q1, q2, q3, q4, q5, q6) " +
                    "VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
                    [r.studentID, r.name,
                     r["1"], r["2"], r["3"], r["4"], r["5"], r["6"]]);
  
We need to change getAllSubmissionsDB(), replacing lines 49-57 (3 changes):

    const query =
          db.prepareQuery(
              "SELECT id, studentID, name, q1, q2, q3, q4, q5, q6 FROM " +
                  table + " ORDER BY name ASC LIMIT 50");

    for (const [id, studentID, name, q1, q2, q3, q4, q5, q6]
         of query.iter()) {
        state.push({id, studentID, name, q1, q2, q3, q4, q5, q6});
    }

And we need to add a line to template_addRecord(), inserting after 137:

    <p>Q5: ${obj.q6}</p>

To make the listing work, we also have to change list.js, inserting into insertTableData() after line 28:

        row.push(rowMarkup(r.q6));

And, of course, we need to add a heading, inserting the following into list.html, after line 21:

          <th>Question 6</th>

Clearly the number of questions is hard-coded into too much of submitdemo.js!

(0.5 point for validator.js, 0.5 for all the changes to submitdemo.js, 0.5 for list.js, and 0.5 for list.html)


4. [2] How could you use a command-line web request tool (such as <tt>wget</tt> or </tt>curl</tt>) to upload data that is not a properly validated submission?

A: Submission uploads are sent to /uploadSubmission as a JSON object that is not validated by the server. (The handling happens in routePost(), then addSubmission(), and then addSubmissionDB() in submitdemo.js.) So, we just need to POST a JSON object to the server using wget or curl. With wget, we could do the following:

  wget --post-data='{"1":" An answer\n\n","2":"","3":"","4":"","5":" more answers\n","6":"","name":"Sarah","studentID":"771572"}' http://localhost:8000/uploadSubmission

This is a valid submission upload (with six questions). But this also works:

  wget --post-data="{}" http://localhost:8000/uploadSubmission

This submits an empty JSON object, which will result in an entry of all null's in the database.

The same works for curl:

  curl -d '{"name":"Jane"}' http://localhost:8000/uploadSubmission

This creates a submission with only the name field completed and the rest null's.

(1 point for valid syntaxt for making a POST and specifying the data, 1 for an example of invalid input.)


5. [4] How could you change submitdemo so that it also validates submissions on the server? The validation should use the same logic as the client-side validator does. When an invalid submission is detected, the server should return the status code 400 and the string "Bad Request".

A: The basic approach to this question is to copy the validator code to the server and have it run there on the raw submission. So we need to 1) make sure the raw submission is uploaded, 2) make the validator run on the server, and 3) return the appropriate error when the submission doesn't validate.

First, copy static/validator.js to validator-server.js. In this copy, delete the following functions and declarations:
  analysisSection
  loadAssignment()
  hideAnalysis()
  uploadSubmission()
  doUploadSubmission()

Next, change updateTag() to the following:

function updateTag(id, val) {
    console.log(id + " update: " + val);
}

Then delete the 13 lines at the end of checkSubmission() that displays the submission, making the end of the function the following:

    questionString = questionList.toString();
    if (questionString !== expectedQuestionList) {
        updateTag("status", "ERROR expected questions " +
                  expectedQuestionList + " but got questions " +
                  questionString);
    } else {
        updateTag("status", "PASSED " +
                  fn + ": " + q.name + " (" + q.studentID + ")");
    }
        
    return q;

Then, add "export" to the checkSubmission() declaration. Now we can treat validator-server.js as a module that can be imported.

Add the following line to the top of submitdemo.js:

  import { checkSubmission } from "./validator-server.js";

Now, to use the validator, we need to send the raw submission and its filename to the server. To do this, we need to change the call to doUploadSubmission() in loadAssignment() (line 29) in validator.js to the following:

  doUploadSubmission(fn, { fn, content } );

It will now upload an object with the filename and file contents.

You could also change it to only upload if the submission is validated:

        if (upload && q) {
            doUploadSubmission(fn, { fn, content } );
        }

Then we have to change the beginning of addSubmission() in submitdemo.js as follows.  First, add the following to the top of the file:

  const status_BAD_REQUEST = 400;

Then modify addSubmission() as follows:

async function addSubmission(req) {
    const submission = await req.json();
    const q = checkSubmission(submission.fn, submission.content);
    
    if (q) {    
	const result = addSubmissionDB(db, table, q);
	var response;
    
	if (result) {
            response = {
		contentType: "text/plain",
		status: status_OK,
		contents: "Got the data",
            };
	} else {
            response = {
		contentType: "text/plain",
		status: status_INTERNAL_SERVER_ERROR,
		contents: "Internal server error"
            };
	}
    } else {
	response = {
            contentType: "text/plain",
            status: status_BAD_REQUEST,
            contents: "Bad Request",
	};
    }
	
    return response;
}

Note the old code is inside of the if clause after the test for checkSubmission(), with the else clause returning the bad request response.

Fortunately the existing validator code handles errors already, so nothing needs to be done there to report when the upload fails.

When developing my solutions, I had a number of errors, ranging from trying to insert the raw submission into the database to crashes for invalid syntax. This one should have taken you some work.


6. [4] How could you change submitdemo so it only allows one upload per student ID? When a second upload with the same ID is attempted, the server should reply with a 409 status code with the text "Conflict" and not add the submission to the database. Upon such an error, the client should update its status to say "ERROR Upload forbidden, student has already submitted."

A: It is possible to solve this by doing a database query to see if the student ID has already been used and then report an error if so. Instead, for this solution we'll add a UNIQUE constraint to the studentID field in the SQLite database, and then we'll check the errors returned in order to report the right kind of error.

First, add the error code to the top of submitdemo.js:

  const status_CONFLICT = 409

Next, add a UNIQUE to the declaration of the studentID field in the table creation inside the CREATE TABLE command at the beginning of submitdemo.js:

    studentID INTEGER UNIQUE,

THen, to check the error properly, we first have to change addSubmissionsDB() to use a try/catch block and to return null when the query succeeds and the error when it doesn't:

function addSubmissionDB(db, table, r) {
    try {
	db.query(`INSERT INTO ${table} ` +
		 "(studentID, name, q1, q2, q3, q4, q5) " +
		 "VALUES (?, ?, ?, ?, ?, ?, ?)",
		 [r.studentID, r.name,
		  r["1"], r["2"], r["3"], r["4"], r["5"]]);
	return null;	  
    } catch (e) {
	return e;	
    }	
}   

And then we need to change addSubmission to check the response for the code 19 error which is the standard error SQLite returns on a constraint error. (There are more specialized errors that can say that the constraint violated was UNIQUE but I couldn't get that to work.)

	const result = addSubmissionDB(db, table, submission);
	var response;

	if (result === null) {
            response = {
		contentType: "text/plain",
		status: status_OK,
		contents: "Got the data",
            };
	} else if (result.code === 19) {
            response = {
		contentType: "text/plain",
		status: status_CONFLICT,
		contents: "Conflict"
            };
	} else {
            response = {
		contentType: "text/plain",
		status: status_INTERNAL_SERVER_ERROR,
		contents: "Internal server error"
            };
	}


7. [4] How could you change the analysis page of submitdemo so it reports the number of empty answers for each question? (Thus, it should report a number for each of the five or six questions in each submission.) Note that an empty answer is one that is the empty string or a string consisting of only whitespace characters.

A: One way to approach this problem would be to make sure when answers were only whitespace they would be entered into the database as null values, as then you can just do queries for null answer fields.

Instead, here I get all the entries from the database and do stats manually on each question, checking to see which ones are blank in the state array of objects.

First, define a function to calculate the raw stats, given the database and table info and returning an object with the per-question counts:

function calcBlankQuestionStats(db, table) {

    // From https://stackoverflow.com/questions/154059/how-do-i-check-for-an-empty-undefined-null-string-in-javascript
    
    function isBlank(str) {
	return (!str || /^\s*$/.test(str));
    }

    var state = getAllSubmissionsDB(db, table);
    const counts = Array(5).fill(0);
    var f, s;

    for (s of state) {	
	for (f=0; f < counts.length; f++) {
	    let qindex = f + 1;
	    let question = "q" + qindex;

	    if (isBlank(s[question])) {
		counts[f]++;
	    }
	}
    }

    return counts;
}


Then add the following to the top of showAnalysis():

    var blankQuestionStats = calcBlankQuestionStats(db, table);
    var blankQuestionList =
	'<li>' + blankQuestionStats.join('</li> <li>') + '</li>';
    

And then add the following to the end of the analysisBody template:

    <p>Blank questions:
      <ol>
       ${blankQuestionList}
      </ol>
    </p>

That's it!