For the Algorithmics course the feature to add time limits to different test cases is required so that we can judge the performance of the solutions submitted by the students. Whereas it is possible to constrain the testcases with certain time limits, it is not yet possible to see how long the reference solution requires to pass the specification tests. As a result one can only determine a correct time limit by applying some form of binary search for every test case for every assignment, obviously this is very tedious and probably not necessary.

Is it possible to show the time required by the reference solution to execute the individual specification tests?

Until this time is available, it is impractical (if not impossible) for TAs to properly set an accurate time limit for the solutions submitted by the students.

Submitted by Stefan Hugtenburg on 7 November 2014 at 13:42

I’ve adapted the specification test with some junit instrumentation. See here: http://weblab.tudelft.nl/assignment/2331/edit#Test-Spec-
In the edit assignment view, when you run the spec tests, you’ll get to see how long it takes to run each test.

This does the trick:

    long time = 0;
    @Rule
    public TestName name = new TestName();
    @Before
    public void setUp() { time = System.currentTimeMillis(); }
    @After
    public void tearDown() { System.out.println("Test '" + name.getMethodName()+ "' took "+ (System.currentTimeMillis()-time) + "ms" ); }

The console will display:

Status: Done
Test 'example' took 7ms
Test 'set1' took 1ms
Test 'set2' took 1ms
Test 'set3' took 3ms
Test 'set4' took 304ms
Test 'set5' took 933ms
Test 'set6' took 16ms
Test 'set7' took 302ms
Test 'set8' took 2ms
Test 'set9' took 3ms
Test 'set10' took 23ms
Test 'set11' took 372ms
Test 'set12' took 2ms
Test 'set13' took 26ms
Test score: 14/14