4.26.3. Setting Up the User Retirement Driver Scripts

Tubular (edx/tubular on github) is a repository of Python 3 scripts designed to plug into various automation tooling. Included in Tubular are two scripts intended to drive the user retirement workflow.

scripts/get_learners_to_retire.py

Generates a list of users that are ready for immediate retirement. Users are “ready” after a certain number of days spent in the PENDING state, specified by the --cool_off_days argument. Produces an output intended for consumption by Jenkins in order to spawn separate downstream builds for each user.

scripts/retire_one_learner.py

Retires the user specified by the --username argument.

These two scripts share a required --config_file argument, which specifies the driver configuration file for your environment (for example, production). This configuration file is a YAML file that contains LMS auth secrets, API URLs, and retirement pipeline stages specific to that environment. Here is an example of a driver configuration file.

client_id: <client ID for the retirement service user>
client_secret: <client secret for the retirement service user>

base_urls:
    lms: https://courses.example.com/
    ecommerce: https://ecommerce.example.com/
    credentials: https://credentials.example.com/

retirement_pipeline:
    - ['RETIRING_EMAIL_LISTS', 'EMAIL_LISTS_COMPLETE', 'LMS', 'retirement_retire_mailings']
    - ['RETIRING_ENROLLMENTS', 'ENROLLMENTS_COMPLETE', 'LMS', 'retirement_unenroll']
    - ['RETIRING_LMS_MISC', 'LMS_MISC_COMPLETE', 'LMS', 'retirement_lms_retire_misc']
    - ['RETIRING_LMS', 'LMS_COMPLETE', 'LMS', 'retirement_lms_retire']

The client_id and client_secret keys contain the oauth credentials. These credentials are simply copied from the output of the create_dot_application management command described in Retirement Service User.

The base_urls section in the configuration file defines the mappings of IDA to base URLs used by the scripts to construct API URLs. Only the LMS is mandatory here, but if any of your pipeline states contain API calls to other services, those services must also be present in the base_urls section.

The retirement_pipeline section defines the steps, state names, and order of execution for each environment. Each item is a list in the form of:

  1. Start state name

  2. End state name

  3. IDA to call against (LMS, ECOMMERCE, or CREDENTIALS currently)

  4. Method name to call in Tubular’s edx_api.py

For example: ['RETIRING_CREDENTIALS', 'CREDENTIALS_COMPLETE', 'CREDENTIALS', 'retire_learner'] will set the user’s state to RETIRING_CREDENTIALS, call a pre-instantiated retire_learner method in the CredentialsApi, then set the user’s state to CREDENTIALS_COMPLETE.

4.26.3.1. Examples

The following are some examples of how to use the driver scripts.

4.26.3.1.1. Set Up Environment

Set up your execution environment.

git clone https://github.com/edx/tubular.git
cd tubular
virtualenv --python=`which python3` venv
source venv/bin/activate

4.26.3.1.2. List of Targeted Learners

Generate a list of learners that are ready for retirement (those learners who have selected and confirmed account deletion and have been in the PENDING state for the time specified cool_off_days).

mkdir learners_to_retire
scripts/get_learners_to_retire.py \
    --config_file=path/to/config.yml \
    --output_dir=learners_to_retire \
    --cool_off_days=5

4.26.3.1.3. Run Retirement Script

After running these commands, the learners_to_retire directory contains several INI files, each containing a single line in the form of USERNAME =<username-of-learner>. Iterate over these files while executing the retire_one_learner.py script on each learner with a command like the following.

scripts/retire_one_learner.py \
    --config_file=path/to/config.yml \
    --username=<username-of-learner-to-retire>

4.26.3.2. Using the Driver Scripts in an Automated Framework

At edX, we call the user retirement scripts from Jenkins jobs on one of our internal Jenkins services. The user retirement driver scripts are intended to be agnostic about which automation framework you use, but they were only fully tested from Jenkins.

For more information about how we execute these scripts at edX, see the following wiki articles:

And check out the Groovy DSL files we use to seed these jobs: