When students write their own Java tests, the only information they get is the number of passed/total tests. Is it possible to give them the same information about their own tests as I get for my specification tests? Of course, they should not see details about specification tests. But without any information about the issues in their own tests, it will be hard to find errors in tests or implementations.

Submitted by Guido Wachsmuth on 12 February 2014 at 16:32

Which details do you mean exactly?
I just hit Run Your Test in a java assignment with errors, it outputs:

Status: Done
driehoekTest(UTest): expected:<12.0> but was:<6.0>
Test score: 0/1

Indeed, this works. What I tried was to change an assertFalse() call into an assertTrue() call and vice versa. Here is one working test case:

@Test
public void testAddNonEmpty2()
{
	SortedList list = new SortedList(-10); 
	assertTrue(list.isEmpty());
	list.add(10);
	assertFalse(list.isEmpty());
}

When I change the first assertion into assertFalse I get a message testAddNonEmpty2(UTest): null.
When I change both assertions in another, similar test, I just get Test score: 15/17 without any details.
I was assuming that this is a general issue, but it seems to be a special one.

Running this test from eclipse, junit only reports that the test failed, and the trace shows an AssertionError with no message.
In the backend, we toString the Failure’s that are reported (javadoc of Failure class).

We might want to change this to use getMessage or add other information such that it becomes clear that it is an AssertionError in this case.

Fixed, now messages are in form of:

testRemoveSize2(UTest) failed: 'java.lang.AssertionError: expected:<4> but was:<3>'
testAddNonEmpty2(UTest) failed: 'java.lang.AssertionError'