Overview: A/B Routing workflow module (Console)

This article provides an overview of the A/B Routing workflow module in Console.

In this article

Overview

The A/B Routing workflow module reroutes a percentage of workflow sessions from the primary route (route B) to a secondary (route A) destination. 

The A/B Routing module is most often used for A/B testing, sometimes referred to as split testing, where one subset of people go through an experimental workflow experience (route A) while the rest of people go through a regular (or “control”) workflow experience (route B). 

A/B testing is useful when you want to improve a workflow experience for customers. You can use data gathered from customers’ workflow sessions and surveys to determine if one branch of the workflow performed better than the other branch. 

In the screenshot below, the example workflow is testing two branches: one where a customer is immediately connected with an agent and one where a customer goes through a Say + Intent experience to engage with a bot before connecting with a queue user. In this example, the business can examine whether customers were better served through one method over the other by concurrently running both workflow experiences with the A/B Routing module. After running the A/B test for a long enough period of time, they can compare the results to determine which experience will be the official route for customers.

Below are some common use cases for using the A/B Routing module:

• Testing new workflow experiences:  Use the A/B routing module to send a percentage of workflow sessions from your primary workflow to an updated, secondary version of the existing workflow. Through reporting, this gives you the ability to collect performance data for the new workflow version by testing the process on a fraction of workflow sessions without fully cutting over to the new workflow process before having the opportunity to measure success rates. 

• Randomize customer satisfaction surveying:  Use the A/B Routing module in a status-based workflow to automatically trigger post-call work for only a percentage of queue interactions instead of all of them. For example, you can use an A/B Routing module to ensure that only a certain percentage of customer interactions receive a customer satisfaction survey. This ensures that a variety of interactions receive a survey - not just interactions that ended positively.

In the section below, we’ll explore how the A/B Routing module works.

How it works

The A/B Routing module can be configured to route a portion of workflow sessions from the B route (primary route) to the A route (secondary route). The number of workflow sessions it routes is dependent on two configurable settings in the module:

• Time Period

• Percentage of sessions

We’ll review each of these settings in more detail below.

The Time Period menu is where you select a window of time to measure. The module counts the number of workflow sessions the workflow processed during that window of time. By counting the number of workflow sessions that took place during that time period, the module creates a sample size to use in conjunction with the Percentage of Sessions field. This is how the A/B Routing module calculates how many users to route down the A path or the B path. 

In other words, the A/B Routing module counts old sessions to predict how many new sessions you’ll have in a workflow and route users accordingly.

The options for the Time Period menu include:

• Yesterday (all of the workflow sessions that ran yesterday)

• Last 7 Days (all of the workflow sessions that ran in the last seven days)

• Last 30 Days (all of the workflow sessions that ran in the last month)

The Percentage of Sessions menu is where you select how many new sessions you want to route down the “A” path. For example, if you select the “1 of 5” option here, then this means that you want 20% of your new workflow sessions to go down the “A” path.

Here are all the available options: 

• 1 in 2 (~50%)

• 1 in 3 (~33%)

• 1 in 4 (~25%)

• 1 in 5 (~20%)

• 1 in 6 (~16%)

• 1 in 7 (~14%)

• 1 in 8 (~12.5%)

• 1 in 9 (~13.3%)

• 1 in 10 (~10%)

Scenario

Let’s look at how this works by walking through a scenario. In the following scenario, this is how the A/B module would work:

• 100 workflow sessions happened during the time period of yesterday

• The percentage of sessions is set to 1 in 10

• Result: 1 out of every 10 (or 10%) newly created workflow sessions will run through the A path (secondary path). The other 90 sessions would go through the B path (primary path).

Note that, while there are only two branches of the A/B Routing module, you can use additional A/B Routing modules to make this logic as simple or as complex as necessary to meet your business requirements.

Visual breakdown

Exterior structure

Interior structure

Below is the deep dive explanation for the interior area of the A/B Routing module.

• Label: The Label field is where you define a name for the module.

• Time Period: The Time Period menu is where you can define the window of time for counting the number of historical workflow sessions that are then used for determining how many new workflow sessions are routed down the A and B path the next day. For example, the “Last 7 Days'' option will count the number of workflow sessions over the last seven days.

When a new workflow session is created in the workflow, identifies how much historical data you want reviewed to determine how many future workflow sessions are routed down either path. This decision is based on the Time Period menu, which could be from the last day, week, or month leading up to when the workflow session was created. This means that the time period set here is based on time relative to the workflow session creation and isn’t based on a specific selection of calendar days. This is an especially important setting for workflows that do not run frequently. 

For example, if you have a workflow that is known to run once every three days, in order to create a large enough data pool to better analyze routing  newly created sessions, set the Time Period menu to ‘Last 30 Days’ in addition to setting your Percentage Of Sessions because there’s more data to review in 30 days as opposed to the last 7 days. 

If you have a workflow that is known to process a high volume of sessions daily, set the Time Period to ‘Yesterday’ as you don’t need as much time span to create a large enough data pool to accurately measure if workflow sessions are being routed accordingly. 

The menu options here are:

• • • Yesterday (all of the workflow sessions that ran yesterday)

    • Last 7 Days (all of the workflow sessions that ran in the last seven days)

    • Last 30 Days (all of the workflow sessions that ran in the last month)

• Percentage Of Sessions: The Percentage Of Sessions menu allows you to define how many newly created inbound workflow sessions route through the primary route (route B) and through the secondary route (route A). 

For example, if you only want to test a new workflow process on 20% of the newly workflow sessions being managed through this workflow, select ‘1 in 5’. This means that four out of every five sessions will go through the primary route (B) while the fifth, remaining workflow session will flow through the secondary route (A). The menu options are:

• • • 1 in 2 (~50%)

    • 1 in 3 (~33%)

    • 1 in 4 (~25%)

    • 1 in 5 (~20%)

    • 1 in 6 (~16%)

    • 1 in 7 (~14%)

    • 1 in 8 (~12.5%)

    • 1 in 9 (~13.3%)

    • 1 in 10 (~10%)