You could always flip a coin.

This is a time-tested method for making a choice between two options. It’s fun to imagine all the high-stakes business plays that have been decided by the old heads-or-tails.

But thanks to A/B testing software, you can make data-based decisions with your advertising, user experience design and more. This form of controlled experiment, also known as split testing, has been a go-to for the world’s most successful companies — and a direct contributor to their success.

An A/B test involves randomly presenting two different versions of something (for example, a sign-up button or a header image) to users or visitors and measuring the performance of each. Any impact on traffic, click-throughs and the like — whether positive or negative — is taken into account when evaluating the options and is used to influence final versioning.

If you’re trying to perfect your website or are planning a hefty performance marketing campaign, these A/B testing experiments allow you to monitor realistic interactions and apply these insights as you see fit. It’s a key factor in how businesses maximize their conversions, turning web surfers into customers, subscribers and so on.

Tools designed for A/B testing fall under the umbrella of conversion rate optimization software. These various technologies work hand-in-hand to chisel your digital presence into a fine-tuned machine. Some simple A/B testing will go a long way in shaping your audience interactions and, in turn, boosting your revenue and reputation.

Thanks to artificial intelligence software and other innovations, controlled experiments like these are becoming easier (and more insightful) than ever. We’re a far way from flipping coins now, and who knows where this technology leads next. With that said, A/B testing remains a practical, DIY strategy for companies of all sizes when it comes to tweaking web applications, homepages and marketing materials for optimal results.

We’ve compiled a handful of A/B testing statistics to help illustrate the popularity and the impact of these tests in the professional landscape. If you find yourself flipping coins and unhappy with the results, you might consider these tests as a logical alternative.

General A/B testing statistics

• Better UX design as a result of user testing can increase a company’s conversion rate by 400 percent. (Entrepreneur, 2018)
• 71 percent of companies run two or more A/B tests a month. (Invesp)
• 77 percent of companies perform A/B tests on their websites. (Invesp)
• 60 percent of companies perform A/B tests on a landing page. (Invesp)
• 59 percent of companies perform A/B tests on emails. (Invesp)
• 58 percent of companies perform A/B tests on paid search campaigns. (Invesp)
• Fewer than half of companies (44 percent) use split testing software. (Invesp)
• 58 percent of companies use A/B testing for conversion rate optimization. (Invesp)
• 60 percent of companies believe A/B testing is highly valuable for conversion rate optimization. (Invesp)

• 63 percent of companies believe it is not difficult to implement A/B testing. (Invesp)
• 7 percent of companies believe it is very difficult to implement A/B testing. (Invesp)
• 1 out of 8 A/B tests has driven significant change for an organization. (Invesp)

A/B testing case studies

• Google ran its first A/B test in the year 2000 to determine the optimum number of search results per page. (Marketing Mag, 2018)
• In 2011, Google ran more than 7,000 A/B tests on its search algorithm. (Wired, 2012)
• Microsoft Bing performs more than 1,000 tests per months. (Marketing Mag, 2018)
• A single ad display change resulting from an A/B test increased Microsoft Bing’s revenue by 12 percent in 2012. (Harvard Business Review, 2017)
• Microsoft Bing’s revenue per search has increased 10 percent to 25 percent each year due to A/B testing. (Harvard Business Review, 2017)
• Amazon, Booking.com, Google and Facebook each conduct more than 10,000 online controlled experiments per year. (Harvard Business Review, 2017)
• At Google and Bing, about 10–20 percent of experiments generate positive results in favor of a new idea being tested. (Harvard Business Review, 2017)
• At Microsoft as a whole, one-third of experiments prove effective, one-third have neutral results, and one-third have negative results. (Harvard Business Review, 2017)
• At Bing, around 80 percent of proposed changes are first run as controlled experiments. (Harvard Business Review, 2017)

• The Obama campaign raised an estimated additional $75 million due to marketing decisions resulting from A/B testing. (Wired, 2012)
• A/B testing during the Obama campaign increased donation conversions by 79 percent. (BrightEdge)

Other A/B testing findings

• 57 percent of experimenters halt an A/B experiment once it appears their original hypothesis was proven, which is also referred to as p-hacking. (Boing Boing, 2018)
• On average, a controlled experiment like A/B testing will need at least 25,000 visitors to reach a statistically significant sample. (VentureBeat, 2018)

Using A/B testing in your business

There’s a lot that goes into a sound marketing strategy. Some real-life feedback certainly helps. A/B testing allows you to solicit organic user feedback by simply observing how different design choices perform. That information enables you to go with the better performer or go back to the drawing board.

There is plenty of data to help make the case for using A/B testing in your business, since this testing method has helped the world’s largest companies — and helped a president get elected, to boot.

Now, testing your visitors without their knowledge demands a certain level of care, and can present ethical challenges depending on the specifics. Case in point: Facebook toying with users’ emotions in 2012 by making news feeds more positive or negative. When developing or considering an A/B test or similar study, you might speak with a corporate law firm or an institutional review board (IRB) before getting started.

Once you've got the green light, A/B testing is easy, thanks to the diverse tools in this software space. Listening to the feedback, though, is what separates the A’s from the B’s.

Want to learn more about A/B testing? Check out our beginner's guide that explains the concept in more detail.