Current Idea

  1. A Peer review is an assignment with:
    1.1 a target assignment, which can be a collection with its subtree
    1.2 type of EssayQuestion or FileSubmissionQuestion, i.e. the review is submitted as text or file.
  2. A Peer review submission can only be started when the deadline and extension of the target assignment is over, i.e. no changes may be made to the target submissions anymore.
  3. When a peer review submission PRSub is started:
    3.1 It picks a random submission from all submissions of the target assignment
    3.1.1 excluding submissions already assigned for review
    3.1.2 excluding submissions where an owner of PRSub is owner of this submission
  4. AC: Peer review submissions can be set as viewable/nonviewable by the owners of the reviewed submission
  5. Grading:
    5.1 Reviews don’t grade the targeted submissions.
    5.2 A Peer review submission itself can be graded using a checklist or grade override.
    5.3 How to grade reviews in a fair way? Reviewing a “perfect submission” is harder than reviewing a really bad one.
  6. Not sure how to deal with sharing of submission data to other students. Should the student give permission for such functionality?
  7. First implementation forces reviewer to view the submission in separate tab, i.e. the review and the to-be-reviewed submission are not shown in a single page.

Problems

@3.1 What conditions should be met in order for a submission to become part of the collection of “to be reviewed”-submissions?

  • A submission may be started, but empty.
  • A submission may be saved, but not submitted

What should (not) be displayed to the reviewer?
Show submission in view mode, but:

  • Hide submission owner(s), which possibly biases the review
  • Hide discussion (?) If not, mask the poster’s names
  • Hide submission data
Submitted by Eelco Visser on 2 January 2013 at 19:55

On 10 January 2013 at 21:00 Eelco Visser removed tag @eelcovisser

On 14 May 2014 at 11:36 Eelco Visser tagged algorithmics

On 14 May 2014 at 11:43 Eelco Visser removed tag algorithmics

On 14 May 2014 at 11:43 Eelco Visser tagged algo-course

On 18 April 2015 at 22:10 Eelco Visser tagged oscar

On 9 November 2015 at 14:23 Elmer van Chastelet tagged sw-security-course

On 9 November 2015 at 14:23 Elmer van Chastelet tagged urgent

On 9 November 2015 at 14:24 Elmer van Chastelet tagged 0.46.1

On 20 November 2015 at 09:34 Elmer van Chastelet removed tag 0.46.1

On 20 November 2015 at 09:34 Elmer van Chastelet tagged 0.46.2

On 25 November 2015 at 12:02 Elmer van Chastelet commented:

moved to issue description


On 26 November 2015 at 07:22 Elmer van Chastelet tagged 0.46.1

On 26 November 2015 at 07:22 Elmer van Chastelet removed tag 0.46.2

On 26 November 2015 at 07:22 Elmer van Chastelet removed tag oscar

On 3 December 2015 at 11:42 Elmer van Chastelet closed this issue.

Log in to post comments