Settings for the course element "Peer Review"
In the course editor, the tabs for the central settings of the course element are at your disposal.
Tab "Title and description"
Here you can enter a title and a description of the course element.
For more information, see the chapter "Title and description".
Tab "Visibility"
Changing the visibility of a course element is one of three ways to restrict access to course elements. If you restrict the visibility of a course element, it will no longer appear in the course navigation.
For more information, see the chapter "Visibility".
Tab "Access"
In the tab "Access" of the course element, you can define who has access to this element.
For more information, see the chapter "Access".
Tab "Learning path"
In case of a learning path course, you can define in the tab "learning path" whether the execution of the course element is obligatory or voluntary. Furthermore, you can define a time period in which the course element can be processed.
The "Learning Path" tab also defines which criterion must be met for the course element to be considered "completed". An overview can be found in the chapter "Learning path".
For more information, see the chapter "Learning path".
Conventional courses do not have the "Learning Path" tab. Instead they have the "Visibility" and "Access" tabs, including expert mode.
Tab "Workflow"
In the first step, the central configuration of the course element takes place in the tab "Workflow". Here you define which "Task" course element you want to use and which steps of the peer review process should be included.
Section "Task"
To conduct a peer review, we need an essay submitted by the course participants. For this purpose, we use the course element "Task".
Once you have added a course element "Task" and set it up with an absolute deadline, you can assign it to the course element "Peer Review".
Section "Peer Review workflow steps"
Here you can activate or deactivate the required workflow steps and set the corresponding deadlines.
- Activate calibration for the participants
- Activate peer review for the participants
- Activate expert review for the experts
- Activate the grading
As we need two different course elements, it is important for these elements to have the same access and visibility rules. Please check the corresponding configuration in case of a conventional course.
When changing the input fields, the fields belonging to the other tabs are not always updated correctly.
A workaround for this update problem is, for example, to click on the course element "Task" and then to return for the configuration of further tabs.
Tab "Review criteria"
The review criteria (comparable to a survey questionnaire) form the pivotal aspect of the entire peer review concept and provide the basis for the assessment strategy.
By clicking the "Create Review Question" button, a new review question can be created. A review question always consists of the question itself and at least two answers. If grading is enabled, each answer option needs a score. We need these scores for further calculations.
Additionally, you can demand the reviewers to justify their answer on every review question. If you select this option, justifying their answer is mandatory for the course participants, and they may not leave the input field empty. Here, you can optionally set a limitation of characters.
As long as the calibration tasks have not yet been finally confirmed and/or the peer review assignment has not yet taken place, the answers on the review questions can be edited.
Review criteria are created within the course element "Peer Review" and not within the course element "Task." This means that—if several tasks are created within the course element "Task"—all tasks use the same review criteria.
Tab "Calibration"
Here the calibration is configured.
- Number of calibrations tasks
- Number of questions: cannot be changed because this number is equal to the number of review questions.
- Number of correct answers: how many questions do the course participants have to answer correctly to achieve the allowed deviation and thus to obtain the points as well.
- Repetitions: here you can configure whether a course participant is allowed to repeat the incorrectly answered calibration tasks (and whether one or two repeats are possible) in case she/he has not answered enough questions correctly (= non-allowed deviation).
When changing the input fields, the fields belonging to the other tabs are not always updated correctly.
A workaround for this update problem is, for example, to click on the course element "Task" and then to return for the configuration of further tabs.
Tab "Reviews"
Section "Peer Review"
If the "Peer Review" workflow step has been activated, you can configure here how many peer reviews the course participants have to complete (from 1 to 5 reviews).
Section "Expert Review"
If the workflow step "Expert Review" has been activated, you can configure here how the assignment of experts has to be done. - Course owners enter themselves as experts - Course owners assign the coaches as experts
Tab "Grading"
This tab is visible in case the evaluation is activated.
Depending on the activated workflow steps, the corresponding sections are shown.
Section "Calibration"
The general calibration settings are configured in the "Calibration" section.
In this section, the assessment specific settings can be found:
- Deduction for repeat: here you can configure the deduction for each repeat.
- Max. score per calibration: cannot be changed because the number is equal to the number of review questions.
- Min. score to pass calibration: how many points needs to be achieved by the participants in order to pass the "Calibration" step. If configured, deductions are also calculated.
Section "Quality of peer Review (grading of completed reviews)"
- Score per Peer Review: maximum number of points (this value can be selected arbitrarily) for each peer review conducted.
- Allowed deviation (full credit): which deviation is allowed to still receive the maximum score.
- Allowed deviation (partial credit): the deviation allowed in order to still receive a portion of the maximum score.
- Deduction (%) of a partial credit: how much is the deduction for an allowed deviation (partial credit).
- Min. score to pass the evaluation: what score needs to be achieved by the course participants to pass the "peer review" step.
Section "Quality of the essay (assessment based on the received reviews)"
- Max. score per peer review received: cannot be changed because the number is equal to the sum of the maximum score for each review question.
- Min. score to pass the evaluation: Which score needs to be achieved by the participant in order to pass as a result of the submitted work.
Section "Self review"
- Max. score of the self review: cannot be changed because the number is equal to the sum of the maximum score for each review question.
- Min. score to pass the evaluation: Which score needs to be achieved by the participant in order to pass as a result of the submitted work.
Section "Expert Review"
- Max. score per expert review: cannot be changed because the number is equal to the sum of the maximum score for each review question.
- Min. score to pass the evaluation: Which score needs to be achieved by the participant in order to pass as a result of the submitted work.
Section "Evaluation strategy of the reviews"
Here you can specify the evaluation strategy of the reviews by defining points or percentages. Each activated workflow step is part of the evaluation.
- Overall evaluation: with points (cannot be changed)
- Calibration (points/percentage): Here you define how large the calibration's share (in points or in percentages) is for the overall evaluation. On this basis, the final grade is calculated.
- Quality of peer review (points/percentage): Here you define how large the quality of the peer review's share (in points or in percentages) is for the overall evaluation. On this basis, the final grade is calculated.
- Quality of the essays (points/percentage): Here you define how large the quality of the essays' share (in points or in percentages) is for the overall evaluation. On this basis, the final grade is calculated.
- Self review (points/percentage): Here you define how large the self review's share (in points or in percentages) is for the overall evaluation. On this basis, the final grade is calculated.
- Expert review (points/percentage): Here you define how large the expert review's share (in points or in percentages) is for the overall evaluation. On this basis, the final grade is calculated.
When changing the input fields, the fields belonging to the other tabs are not always updated correctly.
A workaround for this update problem is, for example, to click on the course element "Task" and then to return for the configuration of further tabs.