8.31.3. Managing Open Response Assessment Assignments

After you publish an open response assessment and learners start to submit responses and perform assessments, there are various actions you can take as a member of the course staff.

In addition, see these other topics about different aspects of open response assessments.

8.31.3.1. View a Specific Learner’s Response and Assessments

You can view the following information about an individual learner’s performance in the open response assessment assignment.

  • The text of the learner’s response, including any files that the learner uploaded.
  • The peer assessments that other learners performed on the learner’s response, including feedback on individual criteria and on the overall response.
  • The peer assessments that the learner performed on other learners’ responses, including feedback on individual criteria and on the overall responses.
  • The learner’s self assessment.
  • The learner’s grade for the assignment.
  • An override grade provided by course staff.

For more details about accessing information for a specific learner, and for an example that shows a learner’s response with peer assessments, see Access a Specific Learner’s Information.

To determine whether a learner has received the required number of assessments from other learners and has completed the required number of assessments for other learners, refer to the Graded By and Must Grade values that were set for the open response assessment assignment in Studio. For more information about these settings, see Specify Step Settings.

8.31.3.1.1. Access a Specific Learner’s Information

Before you can access information about a specific learner’s assignment, you need the learner’s username or email address. For more information, see Download or View Learner Data.

  1. In the course, go to the ORA assignment.

  2. Scroll to the bottom of the problem, then select Manage Individual Learners.

  3. Enter the learner’s username or email address, then select Submit.

    The Manage Individual Learners dialog updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the learner’s response. Only the types of assessment steps (self, peer, or staff) that are included in the assignment are shown.

    If the learner uploaded a file along with her response, select View the file associated with this submission to review or download it.

  4. Select any of the section headings to expand that section.

The expandable sections on the Manage Individual Learners page.

8.31.3.2. Perform Staff Assessments in an ORA Assignment

When a staff assessment step is included in an open response assessment assignment, course team members see a Grade Available Responses button at the bottom of the assignment in the course, and learners see a Staff Assessment step in their assignment.

Note

The scores that you give learners in staff assessment steps override scores from any other assessment type in the assignment, including peer assessments that are completed after the staff assessment.

To perform staff assessments in an assignment, complete these steps.

  1. In the course, go to the ORA assignment.

  2. Scroll to the bottom of the problem, then select Grade Available Responses.

    In the dialog that opens, the number of available and checked out responses is shown. Checked out responses are responses that are currently being graded by you or another course team member.

  3. Select the Staff Assessment heading to open a response that is available for grading.

  4. Perform an evaluation of the response using the problem’s rubric.

  5. Select Submit assessment to submit the assessment and close the grading dialog. Alternatively, select Submit assessment and continue grading to submit the assessment and immediately grade another submission.

    The grade that you have given this learner’s response overrides scores from any other assessment type in the assignment. Peer assessments performed on this response after a staff grade has been given have no effect on the learner’s final assignment grade.

8.31.3.3. Override a Learner’s Assessment Grade

For any open response assessment, whether or not a staff assessment step is already included, you might want to override a learner’s final grade for the assignment. The ability to override the final grade is particularly useful in assignments without staff assessment steps, for example, if a learner’s submission was inappropriately or inadequately graded by peers, or if there are not enough peer reviewers to complete the required number of peer assessments.

Course staff can perform an assessment of an individual learner’s response that acts as an override to the learner’s existing grade. The grade given by staff in the override assessment immediately replaces any other calculated grade as the final grade for the learner.

Note

You can perform override assessments more than once on the same response, regardless of the due date of the assignment. The learner’s final grade on the assessment is updated to reflect the most recent staff override assessment grade.

Learners who have override grades for their submissions see a Staff Assessment step in their assignments, where they can view the rubric and any comments provided in the staff assessments.

8.31.3.3.1. Perform an Override Assessment

To perform an override assessment for a particular learner, complete these steps.

  1. In the course, go to the ORA assignment.

  2. Scroll to the bottom of the problem, then select Manage Individual Learners.

  3. Enter the learner’s username or email, then select Submit.

    The Manage Individual Learners dialog updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the learner’s response.

  4. Select Submit Assessment Grade Override.

  5. Perform an assessment of the learner’s response using the problem’s rubric.

  6. When you have finished the assessment, select Submit assessment.

    The grade that you have given this learner’s response becomes the learner’s final grade on the assignment. Peer assessments are not taken into account in calculating the learner’s final assignment grade when a staff override grade exists.

    Learners who have an override grade for their submission see a Staff Assessment step in their assignment, where they can view the rubric and any comments provided in the staff assessment.

Note

Override assessments can be performed more than once on the same response, regardless of the due date of the assignment. The learner’s final grade on the assessment is updated to reflect the most recent staff override assessment grade.

8.31.3.4. Remove a Learner’s Response from Peer Grading

In a course that contains assignments with peer assessment steps, learners might alert you to vulgar, abusive, or otherwise inappropriate responses that they have seen while performing peer assessments. In such a situation you can locate and remove the submission. Doing so removes the inappropriate response from peer assessments so that it is no longer shown to other learners.

Note

Removing a learner’s submission is an irreversible action.

When you remove an inappropriate submission, the response is immediately taken out of the pool of submissions available for peer assessment. If the inappropriate response has already been sent to other learners for peer assessment, it is also removed from their queue. However, if any learner has already graded the inappropriate response, it is counted as one of the submissions they have graded.

Note

After you remove an inappropriate response from peer assessment, you decide whether the learner who submitted that response is allowed to submit a replacement response. If you do not want to allow the learner to submit a replacement response, you do not need to take any additional action. The learner receives a grade of zero for the entire submission. To allow the learner to resubmit a response for a cancelled submission, you must delete the learner’s state for the problem. For more information, see Delete a Learner’s State for a Problem.

To remove a submission from peer assessment, complete these steps.

  1. Identify the learner who submitted the inappropriate response by following the steps in the Locate a Specific Submission in an ORA Assignment topic.

  2. In the course, go to the ORA assignment that contains the submission you want to remove.

  3. Scroll to the bottom of the problem, then select Manage Individual Learners.

  4. Enter the learner’s username or email, and select Submit.

    The Manage Individual Learners dialog updates with expandable sections for each of the assessment steps in the assignment and other actions you can take on the learner’s response.

  5. Select Remove Submission from Peer Grading.

  6. Enter a comment to explain the removal. This comment appears to the learner when she views her response in the open response assessment problem.

Dialog allowing comments to be entered when removing a learner submission.
  1. Select Remove submission.

    The inappropriate submission is removed from peer assessment. Removed submissions are also removed from the list of Top Responses if they were previously listed.

When you access this learner’s information again by selecting Manage Individual Learners, instead of the response, you see a note showing the date and time that the submission was removed, and the comments that you entered.

In "Manage Individual Learners", the date, time and comment for removal of a learner response is shown instead of the original response.

When the learner views the assignment in the course, she sees that all steps in the assignment have a status of “Cancelled”. Under Your Response, instead of the text of their response, she sees the date and time that their response was cancelled, and the comments relating to the removal of their submission.

In a learner's view of the assignment, all steps have a status of Cancelled, and the learner sees the date, time and comment given for the removal of his submission.

8.31.3.5. Locate a Specific Submission in an ORA Assignment

If you are alerted to an inappropriate ORA submission that you want to cancel and remove from peer assessment, locate the specific submission by following these steps.

  1. Ask the person who reported the incident to send you a sample of text from the inappropriate response.

  2. Contact your edX partner manager to request a data download of ORA responses for your course.

    You will receive the download as a spreadsheet or in .csv file format.

  3. Search the spreadsheet for text that matches the sample text from the inappropriate response.

  4. From any matching entries in the spreadsheet, locate the username of the learner who posted the submission.

  5. Make a note of the username, and follow the steps to remove a learner response from peer grading.