May 09, 2011
At Supercircuits, we run A/B (or split) tests on our marketing emails on a regular basis. Here are some recent ones — see if you can guess which email was the winner for each. Unless indicated otherwise, the email itself was the same and just the subject lines were changed.
This first A/B test was for an email showcasing our infrared security cameras. Which of these emails had the most opens and clicks?
We send out a daily deals email every Monday. Did the incumbent, 2a, keep its title or did the challenger take it down?
Here's an interesting one — for this test we sent a version of the email that included pricing information and one which did not. For both emails we used the same subject line:
This fourth test is for another weekly deals email, that is often themed around a particular product type:
Sometimes small changes can make a big difference. Here are two Fat Tuesday-related email subject lines:
This email is an example of one that goes out to our law enforcement customers:
This last test is part of a reacquisition email campaign we sent out to customers during March Madness. We sent out a series of emails with different offers and subject lines, depending on when you last made a purchase with us.
1. Email 1b was the winner as far as open rates are concerned, with a 7% improvement. However, 1a won as far as overall clicks were concerned with a 6% improvement over 1b.
2. Short and sweet won the day with email 2a having a 27% improvement over 2b for open rates and an 11% improvement in the number of clicks.
3. For some reason, email 3a — which had no prices — had an 8% higher open rate over 3b, even though both subject lines were the same. More importantly, it also had a 60% increase in the number of clicks compared to the email with pricing information.
4. The subject line for email 4a led to a 25% increase in open rates. However, email 4b had a 50% greater number of clicks. I guess the moral of this story is, don't even bother trying to guess the winner … just test.
5. Sometimes small changes also have a small effect. Both email 5a and 5b had the same open rate, with email 5a doing slightly better and generating 4% more clicks.
6. The simpler subject line in email 6b resonated more with readers and generated 72% more opens. It also led to 4x the number of clicks as email 6a (which was exactly the same email apart from the subject line).
7. Subject line 7b was the bigger hit with our target segment, generating 15% more opens and 86% more clicks.
Posted on: May 9, 2011 | 3 Comments