Behind-the-scenes of an Emma split test

Testing subject lines in our monthly newsletter

As the oldest of three girls, my childhood was evenly divided among three categories: eating, sleeping and Barbies. While my sisters acted out elaborate dramas (shocking revelations on the red carpet! a scandalous elopement!) and crafted decorations for our Barbie condos, I generally busied myself sorting, categorizing and organizing Barbie outfits and accessories. In fact, I became a lifelong sorter.

It’s served me well at Emma, where we do our best to practice what we preach, and that includes audience list segmentation. Sure, you can send a monthly blast to all the folks who have ever given you their email addresses, and some of our customers do just fine with this approach. But if you can target audience segments based on their interests, preferences or history with your brand, you’ll create a unique, personal relationship that may just result in brand evangelists.

We’ve got some big plans in the works for better engaging our current customers with variable content, based on their behavior within their accounts. And we send periodic, targeted content to different types of subscribers. At the most basic level, we segment our monthly newsletter list into current customers versus all other subscribers. Just tracking the response differential is really eye-opening. We use our CRM, Salesforce, to identify all of our current customers, then utilize the search and segment feature in Emma to sort out all the newsletter subscribers who aren’t in the Current Customers group — and we put them in a different audience group. Generally speaking, we see an increase in open rates when we’ve identified a reader as an Emma user; moreover, we see an average 10% increase in open rates with readers who are actively using their Emma accounts.

For our July newsletter, we took our response analysis one step further. In addition to splitting our campaigns up between Current Customers and Everybody Else, we did A/B/C testing to see how different subject lines played out. Take a look at the subject lines:

• Version A > An easier way to send campaigns, an email success story and more in Emma’s July Roundup
• Version B > Emma’s July Roundup: An easier way to send campaigns, an email success story and more
• Version C > Click-throughs 10 times the average? How one company did it + much more in Emma’s July Roundup

Before I reveal the results, can you guess which subject line performed the best?

Emma’s handy compare mailings feature made it easy to line up our various versions and see subscriber behavior. Interestingly (or bafflingly, depending on how you look at it), the results weren’t consistent between the two groups. Version A tested higher with our general subscribers, while version C got a better response from our customers. Version B, which was the closest to the standard format of our newsletter subject lines for the last several months, tested the worst with both groups. Perhaps our readers are trying to tell us to mix things up more often?

The only verifiable trend we saw was the one we were already expecting: current Emma users had much higher opens, click-throughs and shares. One version of the campaign showed a whopping 44% open rate. Our active customers were also interested in learning more; an average of 19% clicked through for additional content.

As a lifelong sorter, let me assure you: data and analysis are your (marketing) friends. Use your response statistics and adjust your communications to suit your readers’ interests. Test out different approaches and see how your subscribers respond. And pat yourself on the back when you see your response numbers improve because of segmentation and targeted messaging — that extra works pays off.

Coming soon, we’ll be excited to reveal split testing as a feature right in your Emma account. It’ll make A/B/C subject line testing even easier. Stay tuned for more info on the Emma website and here on the blog. And if you’re just dying to hear more about my childhood adventures in Barbie organization, you’ll have to track me down.

+++++

New to Emma? Give us a try for free.