Email Testing - Why Bother with AB Testing?

Why bother with testing your emails? It seems pretty simple, but when it comes down to tight deadlines and turn around times even the best in the industry tend to skimp on testing time. It truly is a critical step that should not be glanced over when architecting a campaign or optimizing an existing email campaign. Testing also proves extremely valuable when truly driving towards quantifiable goals and business objectives.

Let's take a step back to our elementary school science lessons and talk about AB testing. First, let's define common terms:

Control = to check, test, or verify by evidence or experiments b : to incorporate suitable controls in (a controlled experiment)
Variable = characterized by variations
Constant = marked by firm steadfast resolution or faithfulness : exhibiting constancy of mind or attachment, invariable
(definitions by Webster.com)

Email Variables Include:

• Subject Line


• From Line


• Content


• Creative


• Color


• Number of Links


• CTA (Calls to Action)


• Copy length


• Delivery type: HTML, plain text, etc.


• Price


• Offer


• Frequency


• Time of Day

When performing an AB test you must first select a variable to test and keep all of the other variables set as constants. To achieve the most significantly sound results you want to isolate your variable in order to test each variable independently. You will then hold all of your other variables as constants. When architecting your email campaign be sure to outline how you'll test and keep track of the increase/decrease in effectiveness based on each variable.

Multi-variable testing is another consideration and a subject we'll cover in a future post.