YellowGrass

See which test cases fail in an assignment

For the algoritmics course, the students grade does not scale linearly with the number of test cases they pass.
Instead, the students need to make sure their algorithm is correct and fast enough.
For the TA’s/SA’s to be able to provide help to the students when they get stuck, it would be very helpful to see which test cases they fail (the student doesn’t need to see this).

Submitted by Jesse on 10 November 2014 at 10:40

Issue Log

On 11 November 2014 at 11:12 Elmer van Chastelet tagged urgent
On 11 November 2014 at 11:13 Elmer van Chastelet tagged 0.42.1
On 12 November 2014 at 10:59 Elmer van Chastelet removed tag 0.42.1
On 12 November 2014 at 10:59 Elmer van Chastelet tagged 0.42.2
On 12 November 2014 at 20:59 Mathijs de Weerdt commented:

The students should be allowed to see

  • the number of tests where the time limit is violated
  • the number of tests where the answer was incorrect
  • the number of tests where there was an Exception
On 9 December 2014 at 17:17 Elmer van Chastelet removed tag 0.42.2
On 9 December 2014 at 17:17 Elmer van Chastelet tagged 0.42.3
On 9 December 2014 at 17:26 Elmer van Chastelet removed tag 0.42.3
On 9 December 2014 at 17:26 Elmer van Chastelet tagged 0.42.5
On 16 December 2014 at 10:21 Mathijs de Weerdt commented:

See also https://yellowgrass.org/issue/WebLab/83

On 23 December 2014 at 11:04 Elmer van Chastelet closed this issue.
On 23 December 2014 at 12:19 Elmer van Chastelet commented:

Example of output, when allowed to be viewed by student:

Status: Done
Test score: 0/6


example(UTest) failed: 'org.junit.ComparisonFailure'
set1(UTest) failed: 'org.junit.ComparisonFailure'
set2(UTest) failed: 'org.junit.ComparisonFailure'
set3(UTest) failed: 'org.junit.ComparisonFailure'
set4(UTest) failed: 'java.lang.Exception: test timed out after 1000 milliseconds'
set5(UTest) failed: 'java.lang.Exception: test timed out after 2000 milliseconds'
On 23 December 2014 at 12:22 Elmer van Chastelet commented:

Example of output, when student submission is viewed by instructor:

Status: Done

example(UTest) failed: 'org.junit.ComparisonFailure: expected:<[Delft Jan 1
Delft Tim 3
Gouda Jan 1]> but was:<[ab 1234]>'
set1(UTest) failed: 'org.junit.ComparisonFailure: expected:<[dzekcu fzavii 1
dzekcu qiutco 1
...
tgepck vvubqd 1]> but was:<[ab 1234]>'
set2(UTest) failed: 'org.junit.ComparisonFailure: expected:<a[xnice cxmoks 1
axnice fhmirt 1
....
zxjizo ymfrfk 1]> but was:<a[b 1234]>'
set4(UTest) failed: 'java.lang.Exception: test timed out after 1000 milliseconds'
set5(UTest) failed: 'java.lang.Exception: test timed out after 2000 milliseconds'
Test score: 0/6

Log in to post comments