This topic describes how to beta test your course.
When you are designing a course in Studio, you can use the Preview or View Live options to test the course content, find gaps and errors, and mimic the learner experience. For more information, see Testing Your Course Content.
To gather information about the overall experience of taking a course, you can also conduct a beta test of the course.
During a beta test, you gather information about the experience of taking your course in an effort to improve its quality. The phases of the beta testing process, and some questions that you may find useful to consider as you prepare for each phase, follow.
There is no one answer to any of these questions. They are included here as background on the role that beta testing can play in the preparation of your course.
Beta testers have early access to the course. Beta testers are not members of the course team: they do not have information about “how it is supposed to work”. They use their own computers and Internet connections to view videos, follow links, and complete problems. They interact with the course as learners will to find, and make, mistakes.
However, beta testers are not the same as other learners in the course, either. They have privileged access to the course and have more time to review and complete the course materials than the enrolled learners do. Course discussions are not open before the course start date, so beta testers cannot participate in community conversations. As a result of these differences, beta testers do not receive certificates when they complete a course.
If one of your beta testers wishes to earn a certificate for the course, she must create a separate user account with a different username and email address. She can then use that separate, non-privileged user account to enroll in the course and repeat the work, completing assignments and exams when they are available to all learners.
A beta test is valuable in part because it is unscripted. Your beta testers are not following a predetermined series of steps, or using a particular computer environment, when they access your course. When you recruit beta testers, however, you may find these skills and characteristics to be helpful.
Your beta testers should also have varying levels of knowledge about the course content:
Depending on the objectives you set for your beta test, you might want to consider recruiting testers who use assistive technologies, who have different native languages, or who have varying levels of familiarity with computer software.
The course team can provide valuable feedback about your course. However, they are typically stakeholders in the success of your course and have a significant amount of knowledge about it. As a result, they can be too close to the course to interact with it in the same way as learners will. They can also be either reluctant to provide feedback, or overly zealous.
If you do want a team member to be a beta tester, note that the privileges of the Staff or Admin role override those of a beta tester. The team member must use a different, second email address to create an additional edX account and enroll in the course, and the course team must assign only the beta tester role to that edX account. The course team member experiences the course as a learner only when she signs in to edX using the beta tester account.
Beta testers should interact with everything in the course.
Beta testers can use a desktop computer to access an edX website, the edX mobile app on a smartphone, or both to complete their tests. As they work, beta testers log issues and questions for the course team.
To beta test a course, you:
Define a number of days before content releases for testing to begin.
- For instructor-paced courses, this number applies to the release date for each section and subsection. Beta testers can only access course content the number of days you specify before each section or subsection is released.
- For self-paced courses, this number applies to the course start date. Because sections and subsections for self-paced courses do not have individual release dates, beta testers can access all course content when the beta test begins.
Identify beta testers.
Designated beta testers see course content before learners can. The matrix that follows shows the course content that beta testers can access earlier than other learners in an example instructor-paced course.
|Before the course enrollment date||X|
|Before the course start date||X|
|Before the section release day||X|
|Before the subsection release day||X|
|Before the unit is published||X|
|Before a draft replaces a live unit||X|
The course team can continue to add content in Studio after the beta test begins. When new content is ready for testing, be sure to publish the unit.
When you set up the beta test and define when it starts, verify that the first section, subsection, and unit in your course are available to be tested. If that first unit is not available, an error occurs when testers attempt to access any other content in the course.
To define the start of the beta test, you use Studio to specify a number of days before the course start date for self-paced courses or before the section and subsection release dates for instructor-paced courses. For more information, see How Beta Testers See Course Content.
Beta tester access to course content for an instructor-paced course with Days Early for Beta Users = 20 and 2 sections:
|Course enrollment date = 31 August||31 August; earlier if enrolled by course team|
|Course start date = 15 September||26 August|
|Section 1 release day = 15 September||26 August|
|section 2 release day = 22 September||2 September|
|Subsection 1 release day = 22 September||2 September|
|Subsection 2 release day = 24 September||4 September|
|Subsection 2, unit 1 visibility = public||4 September|
|Subsection 2, unit 2 visibility = public; draft in progress||4 September for public version; no access to draft|
|Subsection 2, unit 3 visibility = private||No access|
In this example, the beta testers cannot access all of the content on the Course page when the beta test starts on 26 August: they can access section 1 but not section 2. You may want to provide a schedule of section availability to your testers so that they can plan their time appropriately. Future release dates do not display to the beta testers in the course.
Before you can add beta testers:
When you add beta testers, note the following.
If you have a number of beta testers that you want to add, you can use the “batch add” option to add them all at once, rather than individually. With this feature, you have options to enroll the beta testers in the course (before or after the Enrollment Start Date) and send an email message to notify the beta testers that they have been added.
To add multiple beta testers:
View the live version of your course.
Select Instructor, and then select Membership.
In the Batch Beta Tester Addition section of the page, enter one or more addresses or usernames separated by commas or line feeds (the Enter key). You can also copy data from a CSV file of email addresses and paste it here.
To enroll the beta testers in your course, leave Auto Enroll selected. If the beta test starts before the course Enrollment Start Date, testers cannot enroll themselves in your course.
To send an email message to the beta testers, leave Notify users by email selected. An example of the message that is sent to a beta tester who is not enrolled in the course follows.
You have been invited to be a beta tester for course at edge.edx.org by a member of the course team.
Visit https://edge.edx.org/courses/course-name/about to join the course and begin the beta test.
Select Add beta testers.
To remove the Beta Tester role from one or more users, enter their email addresses in the Batch Add Beta Testers field and then select Remove beta testers.
The Auto Enroll option has no effect when you select Remove beta testers. The user’s role as a beta tester is removed; course enrollment is not affected.
View the live version of your course.
Select Instructor then Membership.
In the Administration List Management section, use the drop-down list to select Beta Testers.
Under the list of users who currently have that role, enter an email address or username and click Add Beta Tester.
If the beta test starts before the Enrollment Start Date of your course, you can also enroll the beta tester. See Enrollment.
To remove the Beta Tester role from users individually, find the user in the list of beta testers, and then select Revoke access to the right of that user’s email address.
Despite the efforts of the course team and the beta testers, additional problems, questions, and issues can occur while a course is running.