A/B test
Sending two or more versions of an email that differ in one element to parts of a list, and comparing the results to pick the better version.
What it means
Mailchimp's A/B testing lets you test one of four variables: subject line, from name, content or send time. Each version of the variable is a variation, and the test sends the combinations to portions of the audience before choosing a winner by the metric you set.
The usual caution applies to the metric. If the winner is chosen by opens, Apple Mail Privacy Protection and other automated opens can decide it, and Mailchimp warns that bots can inflate clicks too; replies and conversions are less exposed. Small test groups produce differences that are noise.
A test also exposes part of the list to the weaker version, so test changes you would be content to send to everyone.
Related terms
On this site
Sources
What this definition rests on. The wording is ours; the facts are theirs.
- Mailchimp: About A/B testing campaignsmailchimp.com
- Mailchimp: About open and click ratesmailchimp.com
All 432 terms in the glossaryThe wiki: blocklists and bounce codes