A Guide to COS432 Dropbox Reports

When we grade your code, we will run it through a comprehensive suite of automated tests which analyze your program's style and correctness. When you submit your code through Dropbox, your code will be run through a small subset of these tests, and the results will be displayed in your browser. We limit the size of thests you're allowed to see in order to encourage you to design and write your own testing code. It is fine to submit your code any number of times in order to try to pass the Dropbox tests.

The report consists of two major sections, the Assessment Summary and the Assessment Details. The Assessment Details section is further subdivided into Compilation, Style, Findbugs, API, andCorrectness.

Assessment Summary

The report begins with the Assessment Summary. An example is shown below. 
Assessment Summary

Compilation: PASSED
Style: FAILED
API: PASSED

Correctness: 0/1 tests passed

The Compilation, Style, and API tests return either PASSED or FAILED. Compilation returns PASSED as long as javac reports no serious warnings or errors. If Compilation fails, the other tests will not run and you will receive no credit for the submission until your code is resubmitted and passes the test. API ensures that the API of your classes matches the required API in the assignment. Like the compile check, API must be passed for the other tests to run. Style returns PASSED as long as the Checkstyle tool reports no style issues, as detailed more fully below. While we encourage you to fix all Style errors, only especially egregious errors will results in lost points in COS432.

Assessment Details

After the Assessment Summary, the Assessment Details follow; they begin with a listing of the files submitted, as shown below.

Assessment Details

files submitted

----------------------------------
total 136K
-rw-rw-rw-. 1 2.1K Sep 21 12:59 DataSealer.java
-rw-rw-rw-. 1 2.2K Sep 21 12:59 DataUnsealer.java
-rw-rw-rw-. 1 2.5K Sep 21 12:59 PRGen.java
-rw-rw-rw-. 1 1.1K Sep 21 12:59 StreamCryptor.java

Compilation

The Compilation subsection is mostly self explanatory. An example with errors is shown below. 
******************************************************************************
*  compiling
******************************************************************************


% javac PRGen.java
*-----------------------------------------------------------
================================================================

% javac StreamCryptor.java
*-----------------------------------------------------------
StreamCryptor.java:3:  expected
    StdOut.printf("% java StreamCryptor ");
                 ^
StreamCryptor.java:3: illegal start of type
    StdOut.printf("% java StreamCryptor ");
                  ^

The lack of text after the "javac PRGen.java" command indicates that compilation was successful. However, StreamCryptor.java will not compile, and the text shown is the output of javac. If your code does not compile, you should see these same messages when you try to compile on your local machine.

Checkstyle

Checkstyle is a standard tool for ensuring that code obeys some set of style guidelines. The style guidelines for Princeton CS courses can be found here. As style is an inherently subjective enterprise, you may not agree with our guidelines. In COS432, only particularly problematic style errors will result in lost points. 
% checkstyle *.java
*-----------------------------------------------------------
PRGen.java:17:33: Name 'stored_key' must match pattern '^[a-z][a-zA-Z0-9]*$|^[A-Z][A-Z_0-9]*$'.
PRGen.java:12:1: File contains tab characters (this is the first instance).
================================================================
In the example above, there are two style issues. The first is that the variable name stored_key is not in camel case. The second is that the file contains tab characters. To fix the first issue here, you'd simply rename the variable so that it becomes storedKey. To fix the second, you will need to use the text editor of your choice to replace the tabs with spaces.

Findbugs

Like Checkstyle, Findbugs is a standard tool for identifying possible bugs in code. Please see the full list of standard bug patterns for more details. 
% findbugs *.class
*-----------------------------------------------------------
H B ES_COMPARING_STRINGS_WITH_EQ ES: Comparison of String objects using == or != in HorseSorter.sortHorses(int, int)
   At HorseSorter.java:[line 128]
Warnings generated: 1
================================================================
In the example above, two strings are being compared using the == operator as opposed to the .equals() method. Except under limited circumstances, this probably doesn't do what the programmer intended, and thus it is flagged by findbugs (of course, there are times when one might actually want to compare two strings with ==, in which case you would ignore the findbugs warning). The results of findbugs are not counted towards your score.

For some assignments, we may remove findbugs from the autograder due to processing time constraints, but we encourage you to run this tool on your own local machine when debugging your programs.

API

The API test ensures that your program obeys the API given in the assignment specification. 
Testing the APIs of your programs.
*-----------------------------------------------------------
PRGen:
The following fields should be made private:
  *  public byte[] storedKey

The following methods should be removed or made private:
  *  public byte[] outputBytes(int)


StreamCryptor:
The following methods are missing:
  *  public byte cryptByte()


================================================================
In the example above, the PRGen class has two API violations. The first is a public instance of byte[], and the second is a public method called outputBytes, neither of which appear in the assignment specification. In StreamCryptor, we see that the class is missing the cryptByte() method. Until you correct all API violations, your program will not be graded.

Correctness

The Correctness tests ensure that your program behaves as required by the assignment specification. Correctness tests vary widely in their operation and output formatting. An example is shown below. 
Testing methods in DataUnsealer
*-----------------------------------------------------------
Running 1 total tests.

Dropbox Test 1: Ensuring that your DataUnsealer returns null if something goes wrong
  *  Returns null if messages are unsealed in the wrong order
     FAILED: Four messages were sealed, and the 4th was unsealed first. Result should have been null, but was not
==> FAILED

Total: 0/1 tests passed!

For each assignment, you'll need to read the Assessment Details to understand each test. Every test is marked at the bottom as "==> passed" or "==> FAILED". When a test is failed, information is given to the user to assist with debugging. However, this feedback is not always thorough, and may not be sufficient to solve your problem alone. You will often need to write your own tests to uncover the bug.

For many classes, there may be no tests available for you to see. You should still write your own test files.