This topic describes how to work with open response assessments in your edX course.
In an open response assessment, you provide written responses to questions that might not have simple or definitive answers. For some open response assessments, you can submit an image or other file to accompany a written response.
Open response assessments can include a peer assessment, a self assessment, or both.
To assess a response, you compare the response to a rubric that the course team provides. A rubric is a list of expectations that the responses in an open response assessment should meet. Rubrics are made of criteria and options.
The following image shows a rubric with two criteria. Each of the criteria has several options.
When you assess a response, you select the option that best describes the response for each of the criteria.
Some open response assessments provide a Top Responses section that shows the top-scoring responses for the assignment and the scores that these responses received. If provided, this section appears below your score after you complete each step of the assignment.
When you come to an open response assessment in the course, you see the questions and a response field for each question. After you submit your responses, you assess some of your peers’ responses, your own responses, or both, depending on the assignment. You can see the steps that your assignment includes below each response field.
This section describes how you complete an open response assessment that includes a learner training step, a peer assessment, and a self assessment.
At any time during an assessment, you can see your progress at the bottom of the page under Your Grade. A message tells you the steps that you still have to perform before you can receive your grade. For example, you might see the following message.
Not Completed You have not completed the peer assessment step and self assessment step of this problem.
Read each question carefully. Some course teams include important information in the question, such as how long a response must be, or specific topics your response must cover.
The total word count for all responses within the assessment cannot be more than 10,000 words (approximately the equivalent of 20 pages of 8.5x11 inch paper, with text single-spaced).
For each question, type your response into the field under Your Response. When you have completed responses for all the questions, select Submit your response and move to the next step. If you need more time, you can select Save Your Progress to save a draft of your responses, and then come back and submit them later.
After you submit your responses, if other learners have already submitted responses, the peer assessment step starts immediately. However, you do not have to start grading right away. If you want to stop working and come back later, just refresh or reopen your browser when you come back. New peer responses will be available for you to grade.
If no other learners have submitted responses yet, you see the following message.
Waiting for Peer Responses All submitted peer responses have been assessed. Check back later to see if more learners have submitted responses. You'll receive your grade after you complete the peer assessment and self assessment steps, and after your peers have assessed your response.
Note that you can view your own responses at any time after you submit them. To do this, for the response you want to view, select the Your Response heading to expand the response field. Your response appears, along with the status of your response, and information about additional steps you have to complete before you receive your grade.
For some assignments, you might be asked to submit a file along with your text response. Those assignments include Choose File and Upload your file options below the response field. Note the following requirements.
To upload your file, follow these steps.
The name of the file that you selected and uploaded appears below the response field.
You can replace the file that you uploaded with a different one until you submit your response. To do so, follow steps 1-3 again.
As part of an open response assessment, you learn how to assess responses effectively by reviewing and assessing sample responses provided by the course team. You then try to give the sample responses the same scores that the course team selected.
Not all course teams provide sample responses for training. If the course team did not provide sample responses, this step is not included in the assignment.
After you submit your own response, a sample response appears along with the rubric for the assignment. Read the sample response and the rubric carefully, select the options that you think best reflect the response, and then select Compare your selections with the instructor’s selections.
Learning to Assess Responses Your assessment differs from the instructor's assessment of this response. Review the response and consider why the instructor may have assessed it differently. Then, try the assessment again.
For each of the criteria, you see one of the following two messages, depending on whether your selections matched those defined by the course team.
Selected Options Agree The option you selected is the option that the instructor selected.
Selected Options Differ The option you selected is not the option that the instructor selected.
In the following example, the learner chose one correct option and one incorrect option.
You continue to assess the sample response until the options you select for all criteria match the options defined by the course team.
When you have successfully assessed all of the sample responses, you move to the next step in the assignment.
When the peer assessment step starts, you see each original question, another learner’s responses, and the rubric for the assignment. Above the responses you can see how many responses you are expected to assess and how many you have already assessed.
You assess other learners’ responses by selecting options in the rubric, in the same way you assessed the sample responses in the learn to assess responses step. Additionally, this step has a field below the rubric where you can provide comments about the learner’s responses.
Some assessments have an additional Comments field for one or more of the assessment’s individual criteria. You can enter up to 300 characters in these fields. In the following image, both criteria have a Comments field. There is also a field for overall comments on the response.
After you have selected options in the rubric and provided additional comments about the responses, select Submit your assessment and move to response #<number>.
When you submit your assessment of the first learner’s responses, another set of responses opens for you. Assess these responses in the same way that you assessed the first learner’s responses, and then submit your assessment. You will repeat these steps until you have assessed the required number of responses. The number in the upper right corner of the step is updated as you assess each set of responses.
You can assess more peer responses if you want to. After you assess the required number of responses, the step “collapses” so that only the Assess Peers heading is visible.
To assess more responses, select the Assess Peers heading to expand the step, and then select Continue Assessing Peers.
When you have completed the required number of peer assessments, your self assessment opens. You see your responses along with the same rubric that you used in the peer assessment step. Assess your responses, then select Submit Your Assessment.
After you submit your self assessment, if other learners are still assessing your responses, you see the following message under the Assess Your Response step.
Your Grade: Waiting for Peer Assessment Your response is still undergoing peer assessment. After your peers have assessed your response, you'll see their feedback and receive your final grade.
If you see this message, keep checking back periodically until the peer assessments of your work are complete.
When peer assessment is complete, you can see the scores you received from all of the peers who scored your work, as well as your self assessment. You can also see any additional comments that your peers have provided.
If you want to, you can provide feedback on the scores that you received under Provide Feedback on Peer Assessments.
If you have assessed the required number of peer responses and completed your self assessment, you can assess additional peer responses. To do this, select the Assess Peers heading. If any responses remain to be assessed, a new response opens.
Peer assessments are scored by criteria. An individual criterion’s score is the median, not the average, of the scores that each peer assessor gave that criterion. For example, if the Ideas criterion in a peer assessment receives 10 from one learner, 9 from a second learner, and 5 from a third learner, the score for that criterion is 9 (the median), not 8 (the average).
Your final score for a peer assessment is the sum of the median scores for each individual criterion.
For example, a response might receive the following scores from peer assessors.
|Criterion Name||Peer 1||Peer 2||Peer 3||Median|
|Ideas (out of 10)||10||7||8||8|
|Content (out of 10)||7||9||8||8|
|Grammar (out of 5)||4||4||5||4|
To calculate the final score, the system adds the median score for each criterion.
Ideas median (8 out of 10) + Content median (8 out of 10) + Grammar median (4 out of 5) = final score (20 out of 25)
Note, again, that your final score is not the median of the scores that each individual peer assessor gave the response.
If the course team included a Top Responses section, you can review the highest-scoring responses submitted for each question. This section appears only after you have completed all the steps of the assignment.
If a member of the course team deems a response that you have submitted to be inappropriate, she can cancel that response and remove it from peer grading. In the open response assessment you see an indicator that your submission was canceled, with the date and time of the cancellation, and a comment by the course team member about the reason.
The course team member might allow you to submit a replacement response for the canceled one, or she might not. If she does not allow you to submit a replacement response, your grade is zero for the assignment.
If you want to try an example open response assessment problem, check out the edX demonstration course, edX Demo course. In addition to giving you a tour of a typical edX course, the edX Demo course contains information about open response assessments and an example peer assessment.