13. Open Response Assessments

This topic describes how to work with open response assessments in your edX course.

13.1. Overview of Open Response Assessments

In an open response assessment, you provide written responses to questions that might not have simple or definitive answers. For some open response assessments, you can submit an image or other file to accompany a written response.

Open response assessments can include a peer assessment, a self assessment, or both.

  • In the peer assessment, you assess, or grade, responses that several of your peers have submitted, and several of your peers assess your responses.
  • In the self assessment, you assess your own responses.

To assess a response, you compare the response to a rubric that the course team provides. A rubric is a list of expectations that the responses in an open response assessment should meet. Rubrics are made of criteria and options.

  • Criteria describe characteristics that each response should have, such as topics the response should cover.
  • The options for each of the criteria describe how well each response satisfies the criteria.

The following image shows a rubric with two criteria. Each of the criteria has several options.

Rubric showing criteria and options.

When you assess a response, you select the option that best describes the response for each of the criteria.

Some open response assessments provide a Top Responses section that shows the top-scoring responses for the assignment and the scores that these responses received. If provided, this section appears below your score after you complete each step of the assignment.

13.2. Instructions for Learners

When you come to an open response assessment in the course, you see the questions and a response field for each question. After you submit your responses, you assess some of your peers’ responses, your own responses, or both, depending on the assignment. You can see the steps that your assignment includes below each response field.

Open response assessment example with question, response field, and assessment types and status labeled.

This section describes how you complete an open response assessment that includes a learner training step, a peer assessment, and a self assessment.

At any time during an assessment, you can see your progress at the bottom of the page under Your Grade. A message tells you the steps that you still have to perform before you can receive your grade. For example, you might see the following message.

Not Completed

You have not completed the peer assessment step and self
assessment step of this problem.

13.2.1. Submit Your Response

Read each question carefully. Some course teams include important information in the question, such as how long a response must be, or specific topics your response must cover.

Note

The total word count for all responses within the assessment cannot be more than 10,000 words (approximately the equivalent of 20 pages of 8.5x11 inch paper, with text single-spaced).

For each question, type your response into the field under Your Response. When you have completed responses for all the questions, select Submit your response and move to the next step. If you need more time, you can select Save Your Progress to save a draft of your responses, and then come back and submit them later.

After you submit your responses, if other learners have already submitted responses, the peer assessment step starts immediately. However, you do not have to start grading right away. If you want to stop working and come back later, just refresh or reopen your browser when you come back. New peer responses will be available for you to grade.

If no other learners have submitted responses yet, you see the following message.

Waiting for Peer Responses

All submitted peer responses have been assessed.
Check back later to see if more learners have submitted responses. You'll
receive your grade after you complete the peer assessment and self assessment
steps, and after your peers have assessed your response.

Note that you can view your own responses at any time after you submit them. To do this, for the response you want to view, select the Your Response heading to expand the response field. Your response appears, along with the status of your response, and information about additional steps you have to complete before you receive your grade.

Image of the Response field collapsed and then expanded.

13.2.1.1. Submit a File with Your Response

For some assignments, you might be asked to submit a file along with your text response. Those assignments include Choose File and Upload your file options below the response field. Note the following requirements.

  • A text response is required.
  • You can only upload one file to submit with your response.
  • The file that you upload must be smaller than 5 MB in size.
  • Image files can be in .jpg, .gif, or .png format.

To upload your file, follow these steps.

  1. Below the response field, select Choose File.
  2. Select the file that you want to upload, and then select Open.
  3. Select Upload your file.

The name of the file that you selected and uploaded appears below the response field.

Example response with an image of Paris.

You can replace the file that you uploaded with a different one until you submit your response. To do so, follow steps 1-3 again.

13.2.2. Learn to Assess Responses

As part of an open response assessment, you learn how to assess responses effectively by reviewing and assessing sample responses provided by the course team. You then try to give the sample responses the same scores that the course team selected.

Note

Not all course teams provide sample responses for training. If the course team did not provide sample responses, this step is not included in the assignment.

After you submit your own response, a sample response appears along with the rubric for the assignment. Read the sample response and the rubric carefully, select the options that you think best reflect the response, and then select Compare your selections with the instructor’s selections.

  • If all of your selections are the same as the defined selections, the next sample response opens automatically.
  • If any option that you select is not the same as the defined selection, you see the response again, and the following message appears above the response.
Learning to Assess Responses

Your assessment differs from the instructor's
assessment of this response. Review the response and consider why the
instructor may have assessed it differently. Then, try the assessment again.

For each of the criteria, you see one of the following two messages, depending on whether your selections matched those defined by the course team.

Selected Options Agree
The option you selected is the option that the instructor selected.

Selected Options Differ
The option you selected is not the option that the instructor selected.

In the following example, the learner chose one correct option and one incorrect option.

Sample training response, scored.

You continue to assess the sample response until the options you select for all criteria match the options defined by the course team.

When you have successfully assessed all of the sample responses, you move to the next step in the assignment.

13.2.3. Assess Peer Responses

When the peer assessment step starts, you see each original question, another learner’s responses, and the rubric for the assignment. Above the responses you can see how many responses you are expected to assess and how many you have already assessed.

An in-progress peer assessment.

You assess other learners’ responses by selecting options in the rubric, in the same way you assessed the sample responses in the learn to assess responses step. Additionally, this step has a field below the rubric where you can provide comments about the learner’s responses.

Note

Some assessments have an additional Comments field for one or more of the assessment’s individual criteria. You can enter up to 300 characters in these fields. In the following image, both criteria have a Comments field. There is also a field for overall comments on the response.

Rubric with comment fields under each criterion and under overall response.

After you have selected options in the rubric and provided additional comments about the responses, select Submit your assessment and move to response #<number>.

When you submit your assessment of the first learner’s responses, another set of responses opens for you. Assess these responses in the same way that you assessed the first learner’s responses, and then submit your assessment. You will repeat these steps until you have assessed the required number of responses. The number in the upper right corner of the step is updated as you assess each set of responses.

13.2.3.1. Assess Additional Peer Responses

You can assess more peer responses if you want to. After you assess the required number of responses, the step “collapses” so that only the Assess Peers heading is visible.

The peer assessment step with just the heading visible.

To assess more responses, select the Assess Peers heading to expand the step, and then select Continue Assessing Peers.

The peer assessment step expanded so that "Continue Assessing Peers" is visible.

13.2.4. Assess Your Response

When you have completed the required number of peer assessments, your self assessment opens. You see your responses along with the same rubric that you used in the peer assessment step. Assess your responses, then select Submit Your Assessment.

13.2.5. Receive Your Score and Provide Feedback

After you submit your self assessment, if other learners are still assessing your responses, you see the following message under the Assess Your Response step.

Your Grade: Waiting for Peer Assessment

Your response is still undergoing peer assessment. After your peers have
assessed your response, you'll see their feedback and receive your final
grade.

If you see this message, keep checking back periodically until the peer assessments of your work are complete.

When peer assessment is complete, you can see the scores you received from all of the peers who scored your work, as well as your self assessment. You can also see any additional comments that your peers have provided.

A learner's response with peer and self assessment scores.

If you want to, you can provide feedback on the scores that you received under Provide Feedback on Peer Assessments.

13.2.6. Assess Additional Peer Responses (optional)

If you have assessed the required number of peer responses and completed your self assessment, you can assess additional peer responses. To do this, select the Assess Peers heading. If any responses remain to be assessed, a new response opens.

13.3. Peer Assessment Scoring

Peer assessments are scored by criteria. An individual criterion’s score is the median, not the average, of the scores that each peer assessor gave that criterion. For example, if the Ideas criterion in a peer assessment receives 10 from one learner, 9 from a second learner, and 5 from a third learner, the score for that criterion is 9 (the median), not 8 (the average).

Your final score for a peer assessment is the sum of the median scores for each individual criterion.

For example, a response might receive the following scores from peer assessors.

Criterion Name Peer 1 Peer 2 Peer 3 Median
Ideas (out of 10) 10 7 8 8
Content (out of 10) 7 9 8 8
Grammar (out of 5) 4 4 5 4

To calculate the final score, the system adds the median score for each criterion.

Ideas median (8 out of 10) + Content median (8 out of 10) + Grammar median (4 out of 5) = final score (20 out of 25)

Note, again, that your final score is not the median of the scores that each individual peer assessor gave the response.

13.3.1. View Top Responses (optional)

If the course team included a Top Responses section, you can review the highest-scoring responses submitted for each question. This section appears only after you have completed all the steps of the assignment.

Section that shows the text and scores of the top three responses for the assignment.

13.4. Canceled Responses

If a member of the course team deems a response that you have submitted to be inappropriate, she can cancel that response and remove it from peer grading. In the open response assessment you see an indicator that your submission was canceled, with the date and time of the cancellation, and a comment by the course team member about the reason.

The course team member might allow you to submit a replacement response for the canceled one, or she might not. If she does not allow you to submit a replacement response, your grade is zero for the assignment.

13.5. Try an Example Open Response Assessment Problem

If you want to try an example open response assessment problem, check out the edX demonstration course, edX Demo course. In addition to giving you a tour of a typical edX course, the edX Demo course contains information about open response assessments and an example peer assessment.