A tale of two APBs, part deux*

A couple of weeks ago, we sent an Emma APB to our customers letting them know about a new feature in their account that gives them more control over their monthly email sending plans. Just for fun, we sent two version of that campaign – one with a screenshot showing the new feature in action and one without it. At the bottom of each version, we let everybody know there were two versions of the campaign, adding a link to a blog post that featured both side by side.

We were curious to see what impact a screenshot had on people’s involvement with the campaign, and most of us figured that the campaign with the screenshot would boast higher clickthrough rates than its image-less counterpart. After all, a lot of email marketing research and best practices will tell you that images help to grab your readers’ attention and draw ‘em into what you have to say.

To our surprise, though, the overall campaign results for open rates and clickthrough rates were statistically identical between the two versions, almost to the decimal point. At a glance, it looked like our campaign would have been just as well off without that beautiful screenshot. But we also took an in-depth look at the link-by-link breakdown, and that’s where we learned something interesting about what people responded to.

There were two places in the campaign where you could click to log into your account to see the changes in action – one near the top and another in the very last sentence. In the version without the screenshot, the clicks were divided fairly equally between the two, with 55% of clickers opting for the top mention and the remaining 45% preferring the bottom mention.

With a screenshot, though, those numbers changed dramatically. In that case, 80% of people who logged in did so through the top link while 20% used the lower option. It’s not that the screenshot had no effect it all. It’s that it galvanized people to click sooner, having already seen the feature, rather than bother to read through the whole campaign to understand how that feature might work. It didn’t help us get *more* clicks, really, but it did help us encourage people to click sooner.

It’s so important to read all the best practices out there – to see what other organizations have found to be effective for their email campaigns and to learn what your fellow email marketers are talking about. But if you want to answer those burning questions like when to send your campaign, or how many images to use in it, or how long that subject line should be, why not just try two options and see how it goes? If our tale of two APBs has a moral, it’s that the best research you read may be your own.

[tags]email marketing, a/b test, a/b split, myemma.com, emma apb[/tags]

*This is all the French I know.