A/B Testing and Thinking in Bets
Hello Readers! When designing an ad, many variables can influence its effectiveness. How do you know why one ad performed better than another? A/B testing can provide clear insight into what works and what doesn’t.
What is A/B testing?
A/B testing involves creating two versions of the same ad, with half of your audience seeing one version and the other half seeing the second. Once you determine which ad performs better, you can run additional A/B tests or continue running the winning ad. The two ads are identical, except for a single variable being tested.
Constant: All the features that remain the same in both ads.
Variable: The single feature you are testing that differs between the ads.
For example, if you want to test the image within an ad, you will keep everything else the same (description, heading, format, link, etc.) and change only the image.
Why test one thing at a time?
Let’s say you change both the copy and the image in an ad, and ad B performs better. You won’t know whether ad B succeeded because of the copy, the image, or a combination of both. Maybe your audience responds better to longer copy, or perhaps the image is more attention-grabbing. Since you don’t know which element made the difference, you can’t apply that insight to future ads.
What can you A/B test?
While ads are a common example, A/B testing isn’t limited to advertising. Here are some other areas where A/B testing is useful:
Websites: Testing different elements, such as the placement of a CTA button.
Products: Comparing two versions of a product before launching.
Email marketing: Sending two versions of the same email to identify the top performer.
Thinking in Bets
Thinking in Bets by Annie Duke is a book about “making smarter decisions when you don’t have all the facts.” Annie explains how people often conflate their best decisions with the best results and their worst decisions with disastrous outcomes. If life were like chess, this would make sense. However, Annie suggests that life is more like poker, where even the statistically best decision can still lead to an unlucky result.
Adopt a Gray Scale Mindset
Let’s imagine you are completely certain about an opinion you hold. Someone challenges you with, “Wanna bet?” Suddenly, you feel hesitation. Their confidence makes you question your certainty. You begin to see potential flaws in your logic or realize something you may have overlooked.
This is easy to accept with trivial opinions, but when it comes to more significant beliefs, it can feel unsettling and make you defensive. This reaction is natural if you view things in black and white. If you believe you are 100% right, then new information will make you 100% wrong.
By thinking in bets, you leave room for error or luck—for example, saying you are 78% confident but acknowledging the possibility of being wrong. Betting on your beliefs helps you identify gaps in your reasoning and makes you more open to differing opinions. Regardless of whether a decision leads to positive or negative results, the focus should be on improving the decision-making process.
