Pattern: A/B Testing

Comparing multiple versions of something (a feature, new functionality, UI, etc.) underreal customer use conditions quickly gives useful data about which performs better

A/B Testing

A company has a working cloud native infrastructure in place and is aiming to deliver a lot of useful functionality to its customers. Teams are proficient, and all the tech and processes are in place.

In This Context

There is no practical way to predict how customers will respond to changes. In the absence of actual customer usage data, design and implementation decisions must be based on guesswork and intuition. Since our intuition is not perfect and full of biases, we may not get the best possible results.


Prepare multiple versions of a solution to a challenge/problem and present them to randomized small portions of the client base. Measure the customer response in terms of value for them, and based on that choose the solution.


You now have an easy way to test assumptions live in a real-world environment.Instead of making guesses or assumptions regarding which of two ideas, implementation strategies, etc., is better, a team can quickly put together a simple prototype with two or more versions and release them to small subsets of real customers. Based on customer response, the team can choose the more appropriate/preferred solution.This way many options could be tested while costs are ultimately saved because onlythe best option is ever fully implemented.