How does A/B Testing work?

A/B testing is when you have a least two variations of something you want to test. Typically you have a control (which may be the current variant) and then one test variant. You create a hypothesis around the problem you are solving for. You also define your success metrics. You typically run the test until it reaches statistical significance. At this point, most A/B testing software will let you know the test winner and the improvement it may have made. Say for instance, you have a red "buy now" button on your website. You've heard that green buttons work better so you want to test this theory out. The red button is the control variant. You create a variant in which the button is green and this would be variant #1. You set the success metrics to button clicks and decide you only want to send 20% of your traffic the test. The 20% represents a small sample of your traffic which reduces the risk if there's adverse effects of the variant. The test runs until it reaches statistical significance. At this point, statistically there's a winner and we can attribute the changes we made or didn't make as the factor driving the KPI. There can be much more steps and nuances involved. This is the very basic concept of testing.