Getting started with A/B and multivariate testing

ab-testing-emma-guide-tile

The Opportunity

Testing is an optimization service that will enable a brand to …

1. Make informed decisions based on science and data.

2. Produce consistent & systematic improvement.

3. Enable actionable processes.

Why Test?

Testing answers the question why?

  • Pre-testing analysis shows what should be tested by raising questions.
  • Testing provides a systematic and scientific approach to answering these questions.

01 Testing is the process in which we can find actionable learnings by asking (and answering) questions.
By using structured design and evaluation, we can learn what does and doesn’t work with a certain level of confidence.

02 Testing can help us optimize creative, content, frequency, acquisition.
But testing cannot optimize these things automatically.

Testing cannot tell you the best time to send emails, but it can tell you if one time is better than another

Most Tested Areas

Testing Process

A framework for creating a testing plan

Where to Start

Testing Marketing Loop

Defining a Testing Plan

Considering Your Objective

Defining Objective

Creating the Hypothesis

01 What question are we trying to answer?
Establishing the question that needs answered dictates the rest of the test design

This is the most vital part of testing

Designing the Test Type

02 Deciding the type of test to run

Designing the Test Sample Size

Designing the Test Duration

Defining the Control Group

To really understand if a change to an email actually affects response rates, those response rates should be compared to a control group – when possible.

What are the control group options?

Implementation

At this stage, you should know:

  • What you are testing
  • How you are testing
  • Your control group

Evaluating the Results

Testing Repository Part 1

Testing Repository Part 2

Testing Repository Chart

Test Brief Client Example

Testing Repository Chart

What Is Successful Testing?

The Rules for Testing

What Should I Test?

image-content-right-4

Request a demo

Get started with Emma

Emma makes it easier for organizations to send personalized, on-brand campaigns across teams, departments, and locations. With built-in support for consistency and compliance, Emma helps teams scale their email thoughtfully as they grow.

Want to see how
it comes together?