7.26.3. Accessing Assignment and Learner Metrics

After your open response assessment assignment has been released, you can access information about the number of learners in each step of the assignment or the performance of individual learners. This information is available in the Course Staff Information section at the end of each assignment. To access learner information, open the assignment in the courseware, scroll to the bottom of the assignment, and then select the black Course Staff Information banner.

The Course Staff Information banner at the bottom of the peer assessment

When you access a specific learner’s information for an open response assessment, you can view his responses and, if necessary, cancel the learner’s submission so that it is not included in peer assessments.

7.26.3.1. View Metrics for Individual Steps

You can check the number of learners who have completed, or are currently working through, the following steps:

  • Submitted responses.
  • Completed peer assessments.
  • Waiting to assess responses or receive grades.
  • Completed self assessments.
  • Completed the entire assignment.

To find this information, open the assignment in the courseware, scroll to the bottom of the assignment, and then select Course Staff Information.

The Course Staff Information section expands, and you can see the number of learners who are currently working through (but have not completed) each step of the problem.

The Course Staff Information box expanded, showing problem status

7.26.3.2. Access Information for a Specific Learner

You can access information about an individual learner’s performance on a peer assessment assignment, including:

  • The learner’s response.
  • The peer assessments that other learners performed on the learner’s response, including feedback on individual criteria and on the overall response.
  • The peer assessments that the learner performed on other learners’ responses, including feedback on individual criteria and on the overall responses.
  • The learner’s self assessment.

In the following example, you can see the performance information for a specific learner. This learner’s response received one peer assessment, and the learner completed a peer assessment on one other learner’s response. The learner also completed a self assessment.

Report showing information about a learner's response

To determine whether this learner has received the required number of assessments from other learners and has completed the required number of assessments for other learners, refer to the Graded By and Must Grade values that were set for the open response assessment assignment in Studio. For more information about these settings, see Specify Step Settings.

For details about accessing information for a specific learner, and for an example that shows a learner’s response with more assessments, see Access a Specific Learner’s Information.

7.26.3.2.1. Access a Specific Learner’s Information

  1. In the LMS, go to the peer assessment assignment that you want to see.
  2. Scroll to the bottom of the problem, and select the black Course Staff Information banner.
  3. In the Get Student Info box, enter the learner’s username, and select Submit.

The learner’s information appears below the Get Student Info box.

The following example shows:

  • The learner’s response.
  • The two peer assessments for the response.
  • The two peer assessments the learner completed.
  • The learner’s self assessment.

For a larger view, select the image so that it opens by itself in the browser window, and then click anywhere on the image that opens.

Report showing information about a learner's response

7.26.3.3. Remove a Learner’s Response from Peer Grading

If you use open response assessments, learners might alert you to vulgar, abusive, or otherwise inappropriate responses that they have seen while performing peer assessments. In such a situation you can locate and cancel the submission. Doing so removes the inappropriate response from peer assessments so that it is no longer shown to other learners.

Note

Removing a learner’s submission is an irreversible action.

When you cancel an inappropriate submission, the response is immediately removed from the pool of submissions available for peer assessment. If the inappropriate response has already been sent to other learners for peer assessment, it is also removed from their queue. However, if any learner has already graded the inappropriate response, it is counted as one of the submissions they have graded.

Note

After you remove an inappropriate response from peer assessment, you decide whether the learner who submitted that response is allowed to submit a replacement response. If you do not want to allow the learner to submit a replacement response, you do not need to take any additional action. The learner receives a grade of zero for the entire submission. To allow the learner to resubmit a response for a cancelled submission, reset the learner’s attempts for the problem.

Remove a submission from peer assessment by completing these steps.

  1. In the LMS, go to the peer assessment assignment that contains the submission you want to remove.

  2. Scroll to the bottom of the problem, then select the black Course Staff Information banner.

  3. Scroll down to the Get Student Info box, enter the learner’s username in the box, and select Submit.

    The learner’s information appears below the Get Student Info box.

  4. Scroll down to the Student Response section and locate the submission you want to remove.

Dialog allowing comments to be entered when removing a learner submission

  1. Enter a comment to document or explain the removal. This comment appears to the learner when she views her response in the open response assessment problem.

  2. Click Remove submission.

    The inappropriate submission is removed from peer assessment. When you access this learner’s information again, instead of the response, you see a note showing the date and time that the submission was removed, and the comments that you entered.

    Removed submissions are also removed from the list of Top Responses if they were previously listed.

The date, time and comment for removal of a learner response is shown instead of the original response.

7.26.3.4. Locate a Specific Submission in an ORA Assignment

If you are alerted to an inappropriate ORA submission that you want to cancel and remove from peer assessment, locate the specific submission by following these steps.

  1. Ask the person who reported the incident to send you a sample of text from the inappropriate post.

  2. Contact your edX program manager to request a data download of ORA responses for your course.

    You will receive the download as a spreadsheet or in .csv file format.

  3. Search the spreadsheet for text that matches the sample text from the inappropriate post.

  4. From any matching entries in the spreadsheet, locate the username of the learner who posted the submission.

  5. Make a note of the username, and follow the steps to remove a learner response from peer grading.