13. Open Response Assessments¶
In an open response assessment, you provide written responses to questions that might not have simple or definitive answers. For some open response assessments, you can submit an image to accompany a written response.
Open response assessments can include a peer assessment, a self assessment, or both. With a peer assessment, you assess, or grade, responses that several of your peers have submitted, and several of your peers assess your responses. With a self assessment, you assess your own responses. To assess a response, you compare the response to a rubric that the course team provides.
A rubric is a list of expectations that the responses in an open response assessment should meet. Rubrics are made of criteria and options. Criteria describe characteristics that each response should have, such as topics the response should cover. The options for each of the criteria describe how well each response satisfies the criteria. In the following image, you can see a rubric with two criteria. Each of the criteria has several options.
When you assess a response, you select the option that best describes the response for each of the criteria.
Some courses provide a Top Responses section that shows the top-scoring responses for the assignment and the scores that these responses received. If your course team creates this section, you can see it below your score after you complete each step of the assignment.
13.1. Student Instructions¶
When you come to an open response assessment in the course, you will see the questions and a response field for each question. After you submit your responses, you will assess some of your peers’ responses, your own responses, or both, depending on the assignment. You can see the steps that your assignment includes below each response field.
Here, we will walk you through the process of completing an open response assessment that includes a student training step, a peer assessment, and a self assessment.
- Submit your responses to the questions.
- Learn to assess responses.
- Assess responses that other students have submitted.
- Assess your own responses to the questions.
- Receive your score and provide feedback on the peer assessment.
At any time during the assessment, you can see your progress through the assignment at the bottom of the page under Your Grade. A message tells you the steps that you still have to perform before you can receive your grade. For example, you might see the following message:
Not Completed
You have not completed the peer assessment step and self assessment step of this problem.
13.1.1. Submit Your Response¶
Read each question carefully. Some course teams include important information in the question, such as how long a response must be, or specific topics your response must cover.
Note
The total word count for all responses within the assessment cannot be more than 10,000 words (approximately the equivalent of 20 pages of 8.5x11 paper, with text single-spaced).
For each question, type your response into the field under Your Response. When you have completed responses for all the questions, click Submit your response and move to the next step. If you don’t want to finish your response all at once, you can click Save Your Progress to save a draft of your responses, and then come back and submit them later.
After you submit your responses, if other students have already submitted responses, the peer assessment step starts immediately. However, you don’t have to start grading right away. If you want to stop working and come back later, just refresh or reopen your browser when you come back. New peer responses will be available for you to grade.
If no other students have submitted responses yet, you see the following message.
Waiting for Peer Responses
All submitted peer responses have been assessed. Check back later to see if more students
have submitted responses. You'll receive your grade after you complete the peer assessment
and self assessment steps, and after your peers have assessed your response.
Note that you can view your own responses at any time after you submit them. To do this, for the response you want to view, click the Your Response heading to expand the response field. You’ll see your response, the status of your response, and information about additional steps you have to complete before you receive your grade.
13.1.1.1. Submit an Image with Your Response¶
Some assignments ask you to submit an image with your text response. If you can submit an image, you will see buttons that you use to upload your image. You can only upload one image in each response, and your response must include some text along with the image. You cannot submit a response that does not contain any text.
Note
The image file must be a .jpg or .png file, and it must be smaller than 5 MB in size.
To upload your image, complete these steps.
- Click Choose File.
- In the dialog box that opens, select the file that you want, and then click Open.
- When the dialog box closes, click Upload Your Image.
Your image appears below the response field, and the name of the image file appears next to the Choose File button. If you want to change the image, follow steps 1-3 again. You can only upload one image per response.
13.1.2. Learn to Assess Responses¶
In this step, you learn to assess responses effectively by reviewing and assessing sample responses provided by the course team. You try to select the same options for the response as the course team selected.
Note
Not all course teams provide sample responses for training. If the course team did provide sample responses, this step does not appear in the assignment.
After you submit your response, one of the sample responses opens, together with the rubric for the assignment. Read the sample response and the rubric carefully, select the options that you think best reflect the response, and then click Compare your selections with the instructor’s selections.
- If all of your selections are the same as the defined selections, the next sample response opens automatically.
- If any option that you select is not the same as the defined selection, you see the response again, and the following message appears above the response.
Learning to Assess Responses
Your assessment differs from the instructor's assessment of this response. Review the
response and consider why the instructor may have assessed it differently. Then, try
the assessment again.
For each of the criteria, you see one of the following two messages, depending on whether your selections matched those defined by the course team.
Selected Options Agree
The option you selected is the option that the instructor selected.
Selected Options Differ
The option you selected is not the option that the instructor selected.
In the following example, the student chose one correct option and one incorrect option.
You continue to assess the sample response until the options you select for all criteria match the options defined by the course team.
When you have successfully assessed all of the sample responses, you move to the next step in the assignment.
13.1.3. Assess Peer Responses¶
When the peer assessment step starts, you see each original question, another student’s responses, and the rubric for the assignment. Above the responses you can see how many responses you are expected to assess and how many you have already assessed.
You assess other students’ responses by selecting options in the rubric, in the same way you assessed the sample responses in the “learn to assess responses” step. Additionally, this step has a field below the rubric where you can provide comments about the student’s responses.
Note
Some assessments have an additional Comments field for one or more of the assessment’s individual criteria. You can enter up to 300 characters in these fields. In the following image, both criteria have a Comments field. There is also a field for overall comments on the response.
After you have selected options in the rubric and provided additional comments about the responses, click Submit your assessment and move to response #<number>.
When you submit your assessment of the first student’s responses, another set of responses opens for you. Assess these responses in the same way that you assessed the first student’s responses, and then submit your assessment. You will repeat these steps until you have assessed the required number of responses. The number in the upper-right corner of the step is updated as you assess each set of responses.
13.1.3.1. Assess Additional Peer Responses¶
You can assess more peer responses if you want to. After you assess the required number of responses, the step “collapses” so that only the Assess Peers heading is visible.
To assess more responses, click the Assess Peers heading to expand the step, then click Continue Assessing Peers.
13.1.4. Assess Your Response¶
When you have completed the required number of peer assessments, your self assessment opens. You see your responses along with the same rubric that you used in the peer assessment step. Assess your responses, then click Submit Your Assessment.
13.1.5. Receive Your Score and Provide Feedback¶
After you submit your self assessment, if other students are still assessing your responses, you see the following message under the Assess Your Response step.
Your Grade: Waiting for Peer Assessment
Your response is still undergoing peer assessment. After your peers have assessed your
response, you'll see their feedback and receive your final grade.
If you see this message, keep checking back periodically until the peer assessments of your work are complete.
When peer assessment is complete, you can see the scores you received from all of your peers, as well as your self assessment. You can also see any additional comments that your peers have provided.
If you want to, you can provide feedback on the scores that you received under Provide Feedback on Peer Assessments.
13.1.6. Assess Additional Peer Responses (optional)¶
If you have assessed the required number of peer responses and completed your self assessment, you can assess additional peer responses. To do this, click the Assess Peers heading. If any responses remain to be assessed, a new response opens.
13.2. Peer Assessment Scoring¶
Peer assessments are scored by criteria. An individual criterion’s score is the median, not average, of the scores that each peer assessor gave that criterion. For example, if the Ideas criterion in a peer assessment receives a 10 from one student, a 7 from a second student, and an 8 from a third student, the Ideas criterion’s score is 8.
Your final score for a peer assessment is the sum of the median scores for each individual criterion.
For example, a response might receive the following scores from peer assessors.
| Criterion Name | Peer 1 | Peer 2 | Peer 3 | Median |
|---|---|---|---|---|
| Ideas (out of 10) | 10 | 7 | 8 | 8 |
| Content (out of 10) | 7 | 9 | 8 | 8 |
| Grammar (out of 5) | 4 | 4 | 5 | 4 |
To calculate the final score, the system adds the median scores for each criterion:
Ideas median (8/10) + Content median (8/10) + Grammar median (4/5) = final score (20/25)
Note, again, that your final score is not the median of the scores that each individual peer assessor gave the response.
13.2.1. View Top Responses (optional)¶
If the course team included a Top Responses section, you can see the highest-scoring responses that your peers have submitted. This section only appears after you have completed all the steps of the assignment.
13.3. Cancelled Responses¶
If a member of the course team deems a response that you have submitted to be inappropriate, she can cancel that response and remove it from peer grading. In the open response assessment you see an indicator that your submission was cancelled, with the date and time of the cancellation, and a comment by the course team member about the reason.
The course team member might allow you to submit a replacement response for the cancelled one, or she might not. If she does not allow you to submit a replacement response, your grade is zero for the assignment.
13.4. Try an Example Open Response Assessment Problem¶
If you want to try an example open response assessment problem, check out the edX demonstration course, DemoX. In addition to giving you a tour of a typical edX course, DemoX contains information about open response assessments and an example peer assessment.