When a course is being designed in Studio, the course team can choose View Live to assess their progress in designing the course, test their work, find gaps and errors, and mimic the student experience. To gather information about the overall experience of taking a course, you can also conduct a beta test of the course.
During a beta test, you gather information about the experience of taking your course in an effort to improve its quality. The phases of the beta testing process, and some questions that you may find useful to consider as you prepare for each phase, follow.
There is no one answer to any of these questions. They are included here as background on the role that beta testing can play in the preparation of your course.
Beta testers have early access to the course. Beta testers are not members of the course team: they do not have information about “how it is supposed to work”. They use their own computers and Internet connections to view videos, follow links, and complete problems. They interact with the course as students will to find, and make, mistakes.
However, beta testers are not the same as other students in the course, either. They have privileged access to the course and have more time to review and complete the course materials than the enrolled students do. Course discussions are not open before the course start date, so beta testers cannot participate in community conversations. As a result of these differences, beta testers do not receive certificates when they complete a course.
If one of your beta testers wishes to earn a certificate for the course, she must create a separate user account with a different username and email address. She can then use that separate, non-privileged user account to enroll in the course and repeat the work, completing assignments and exams when they are available to all students.
A beta test is valuable in part because it is unscripted. Your beta testers are not following a predetermined series of steps, or using a particular computer environment, when they access your course. When you recruit beta testers, however, you may find these skills and characteristics to be helpful.
Your beta testers should also have varying levels of knowledge about the course content:
Depending on the objectives you set for your beta test, you might want to consider recruiting testers who use assistive technologies, who have different native languages, or who have varying levels of familiarity with computer software.
The course team can provide valuable feedback about your course. However, they are typically stakeholders in the success of your course and have a significant amount of knowledge about it. As a result, they can be too close to the course to interact with it in the same way as students will. They can also be either reluctant to provide feedback, or overly zealous.
If you do want a team member to be a beta tester, note that the privileges of the Staff or Admin role override those of a beta tester. The team member must use a different, second email address to create an additional edX account and enroll in the course, and the course team must assign only the beta tester role to that edX account. The course team member experiences the course as a learner only when she signs in to edX using the beta tester account.
Beta testers should interact with everything in the course.
Beta testers can use a desktop computer to access an edX website, the edX mobile app on a smartphone, or both to complete their tests. As they work, beta testers log issues and questions for the course team.
To beta test a course, you:
Designated beta testers see course content before students can. The matrix that follows shows the course content that beta testers can access earlier than other students.
|Before the Course Enrollment Date||X|
|Before the Course Start Date||X|
|Before the section Release Day||X|
|Before the subsection Release Day||X|
|Before the unit is Published||X|
|Before a draft replaces a live unit||X|
The course team can continue to add content in Studio after the beta test begins. When new content is ready for testing, be sure to publish the unit.
When you set up the beta test and define when it starts, verify that the first section, subsection, and unit in your course are available to be tested. If that first unit is not available, an error occurs when testers attempt to access any other content in the course.
To define the start of the beta test, you use Studio to specify a number of days before the release day that is defined for each section and subsection in your course.
Beta tester access to courseware for a course with Days Early for Beta Users = 20 and 2 sections:
|Course Enrollment Date = 31 August||31 August; earlier if enrolled by course team|
|Course Start Date = 15 September||26 August|
|section 1 Release Day = 15 September||26 August|
|section 2 Release Day = 22 September||2 September|
|subsection 1 Release Day = 22 September||2 September|
|subsection 2 Release Day = 24 September||4 September|
|subsection 2, unit 1 Visibility = Public||4 September|
|subsection 2, unit 2 Visibility = Public; draft in progress||4 September for Public version; No access to draft|
|subsection 2, unit 3 Visibility = Private||No access|
In this example, the beta testers cannot access all of the courseware when the beta test starts on 26 August: they can access section 1 but not section 2. You may want to provide a schedule of section availability to your testers so that they can plan their time appropriately. Future release dates do not display to the beta testers in the courseware.
Before you can add beta testers:
When you add beta testers, note the following.
If you have a number of beta testers that you want to add, you can use the “batch add” option to add them all at once, rather than individually. With this feature, you have options to enroll the beta testers in the course (before or after the Enrollment Start Date) and send an email message to notify the beta testers that they have been added.
To add multiple beta testers:
To remove the Beta Tester role from one or more users, enter their email addresses in the Batch Add Beta Testers field and then click Remove beta testers.
The Auto Enroll option has no effect when you click Remove beta testers. The user’s role as a beta tester is removed; course enrollment is not affected.
To add a single beta tester:
View the live version of your course.
Click Instructor then Membership.
In the Administration List Management section, use the drop-down list to select Beta Testers.
Under the list of users who currently have that role, enter an email address or username and click Add Beta Tester.
If the beta test starts before the Enrollment Start Date of your course, you can also enroll the beta tester. See Enrollment.
To remove the Beta Tester role from users individually, find the user in the list of beta testers, and then click Revoke access to the right of that user’s email address.
Despite the efforts of the course team and the beta testers, additional problems, questions, and issues can occur while a course is running.