{"id":26457,"date":"2018-06-26T02:00:33","date_gmt":"2018-06-25T18:00:33","guid":{"rendered":"https://mercurylabsmarketing.com/?p=26457"},"modified":"2026-03-27T12:08:10","modified_gmt":"2026-03-27T16:08:10","slug":"math-behind-ab-testing-visual-guide","status":"publish","type":"post","link":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/","title":{"rendered":"The Math Behind A/B Testing: A (Simplified) Visual Guide [INFOGRAPHIC]"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Just off the top of your head, can you explain what p-values are and what they tell you about your A/B test results? If reading that question almost made you close the current tab, you\u2019re not alone. In fact, according to FiveThirtyEight, even scientists themselves can\u2019t clearly explain </span><a href=\"https://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"font-weight: 400;\">what p-values really mean</span></a><span style=\"font-weight: 400;\">.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">While your testing tool probably does most of the number-crunching under the hood, it\u2019s much better to gain a solid understanding of how the math works out than simply running the test and waiting for the results to flash on your dashboard.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Without a good grasp of the math and statistical concepts behind A/B testing, you\u2019ll simply end up guessing rather than actually experimenting. Knowing the underlying technical details helps you make better decisions in your testing and sampling design, as well as gives you a sound framework for interpreting results.</span></p>\n\n\n<div id=\"cta_2boi\" class=\"align-center\">\n                <h4 class=\"heading-lg\">Tired of chasing unqualified prospects with no results? </h4>\n                <div class=\"call-to-action-btn\">\n                    <a class=\"wp-block-button__link\" href=\"https://meetings-na2.hubspot.com/callbox-customer-success/us?utm_source=aaILcta\" target=\"_blank\">Start Lead Gen Plan </a>\n                </div>\n            </div>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">That\u2019s why we put together this quick infographic that visually summarizes the ideas behind key mathematical/statistical concepts you typically encounter in A/B testing.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Don\u2019t worry if you\u2019re allergic to equations and arcane Greek symbols. We won\u2019t be dealing with any of those here. We\u2019ll only focus on building the intuition needed to make informed testing decisions.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Also, the rest of the blog post gives a more in-depth discussion of what\u2019s on the infographic, so you should definitely check that out as well.</span></p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"1609\" src=\"https://mercurylabsmarketing.com/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1.webp\" alt=\"The Math Behind A/B Testing: A (Simplified) Visual Guide [INFOGRAPHIC]\" class=\"wp-image-86164\" srcset=\"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1.webp 600w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-112x300.webp 112w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-384x1030.webp 384w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-573x1536.webp 573w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-4x12.webp 4w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-559x1500.webp 559w, https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2023/02/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide-INFOGRAPHIC-1-263x705.webp 263w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" /></figure>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Let\u2019s say you wanted to see if changing the color of your call-to-action (CTA) button on your whitepaper landing page from red to green would impact the number of downloads. You then randomly split your traffic 50-50, with one half assigned to the page having the red-colored CTA (the control group) and the other half assigned to the page which has the green-colored CTA (the variation group).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">After recording 500 unique visits for each page, you observe that the conversion rate (number of downloads as a percentage of page traffic) for the control group was 7%, while the conversion rate for the variation group was 9%. You may be tempted to conclude that changing the CTA\u2019s color has a real impact on conversions. But before you accept the results as valid, you first need to carefully answer a number of questions about your findings, such as:</span></p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Do I have enough samples (page views) for each of the two groups?</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">How likely is it that I got the test results simply by chance?</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">Is the difference between the conversion rates big enough to justify making the change?</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">If I ran the test again and again, how confident am I that it\u2019s going to give me similar results?</span></li>\n</ul>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">These are only a few of the things you need to think about when planning and carrying out A/B tests. Below, we\u2019ll go over the mathematical/statistical tools to help us objectively answer each of these questions.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\">Learn how to <a href=\"https://www.linkedin.com/pulse/ab-test-way-you-should-get-new-leads-callbox-north-america-fhxdc/\" target=\"_blank\" rel=\"noreferrer noopener\">generate leads with A/B testing</a> technique.</p>\n\n\n\n<h2 class=\"wp-block-heading cb-heading-primary\" id=\"h-the-very-basics\">The Very Basics</h2>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">ConversionXL says the three statistical </span><a href=\"https://conversionxl.com/blog/ab-testing-guide/\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"font-weight: 400;\">building blocks of A/B testing</span></a><span style=\"font-weight: 400;\"> are: the mean, variance, and sampling. Let\u2019s now gain a more intuitive feel for these concepts and understand what the numbers really tell us.</span></p>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-mean\">Mean</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">We commonly refer to the mean as the average. But what does the \u201cmean\u201d really mean? You can think of the mean as the number that represents a collection of numbers really well. That is, knowing the mean gives you a rough idea of what values a sample tends to have since most of the numbers in that sample will tend to cluster around the mean.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">For example, if you determined that your average monthly site visits was 70,000 for the last 12 months, then you\u2019re saying that 70,000 is a fairly acceptable summary for your monthly site traffic. That is, most of the time, your monthly site traffic will be \u201cclose\u201d to 70,000.</span></p>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-variance-and-standard-deviation\">Variance and Standard Deviation</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">The variance measures how dispersed the values in a collection of numbers are. The higher the variance, the more scattered the values in our sample set will be.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">As the variance increases, the mean becomes a less reliable representative of our dataset.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Let\u2019s say you want to compare the average time (in seconds) spent on two different pages of your site. For simplicity, you only collect 8 observations for each page. Your datasets look like this:</span></p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\n<pre>[20, 22, 21, 20, 20, 19, 17, 21]</pre>\n</li>\n\n\n\n<li>\n<pre>[14, 27, 31, 10, 11, 28, 2, 37]</pre>\n</li>\n</ol>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Both sample sets have a mean of 20 seconds. It\u2019s easy to see that the average summarizes the Page A sample really well. Most of Page A\u2019s observations tend to stick very closely to 20 seconds. On the other hand, 20 isn\u2019t a very good summary of the observations reported under page B. That\u2019s because the values in the second set tend to be farther away from the mean. If we compute the variance for each of the two sets, we get 2.3 for Page A and 152 for Page B.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">We see that it can be misleading to solely rely on the mean to describe a sample set. That\u2019s why you always need to look at the associated variance as well.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">But one problem with the variance is that its value can be a bit tricky to interpret and use. Just look at the variances we calculated earlier. They\u2019re both expressed in \u201csecond-squared\u201d units (whatever that means).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">To work around this, we often include the standard deviation in our analysis. The standard deviation is simply the square-root of the variance (don\u2019t worry, you don\u2019t need to compute this yourself most of the time).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">As shown in the next example, it\u2019s easier to work with the standard deviation. The standard deviation of the average time spent on Page A is around 1.5 seconds. Now, we can measure how far a given value is from the mean by expressing the difference as units of standard deviation. For example, the value 17 is around 2 standard deviations below the mean of 20 (the difference between 17 and 20 is 3, and 3 divided by the standard deviation of 1.5 is 2).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">The key thing to remember is that the variance tells you how spread out your observations are, and the standard deviation gives you the average distance of each observation from the mean.</span></p>\n\n\n<div id=\"cta_kBOR\" class=\"align-center\">\n                <h4 class=\"heading-lg\">Having hard time to find your target decision-makers? Let us help you! </h4>\n                <div class=\"call-to-action-btn\">\n                    <a class=\"wp-block-button__link\" href=\"https://meetings.hubspot.com/callbox-customer-success/us/?utm_source=aaILnam\" target=\"_self\">Talk to an expert </a>\n                </div>\n            </div>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-sampling\">Sampling</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In our landing page split-test example, we use a sample of 500 unique page visits for each version. We select a sample that (hopefully) statistically represents the entire population of our landing page visitors. Since studying the whole population of page visitors is impractical, we settle for a representative sample instead.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Exactly how large our sample size should be will depend on a number of factors. Although you don\u2019t need to know the formula for computing the ideal sample size, it\u2019s important to understand that it uses the following factors:</span></p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Significance level (the probability of &nbsp;seeing an effect when none actually exists)</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">Statistical power (the probability of seeing an effect when the effect actually exists)</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">Effect size (how big the difference or change is)</span></li>\n</ul>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">We\u2019ll dig deeper into each of these later. For now, the main thing to know is that generally, the larger our sample size is, the more reliable (unbiased) the mean becomes.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\">Discover the <a href=\"https://mercurylabsmarketing.com/lead-generation/strategies-for-bringing-new-clients/?utm_source=aaILnam\">proven strategies that will bring new clients</a>.</p>\n\n\n\n<h2 class=\"wp-block-heading cb-heading-primary\" id=\"h-null-hypothesis-testing\">Null Hypothesis Testing</h2>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">When running A/B tests, we\u2019re actually applying a process called null hypothesis testing (NHT). We compare the conversion rates of the two landing pages and test the null hypothesis that there is no difference between the two conversion rates (meaning the 2-percentage-point difference between the control\u2019s 7% and the variation\u2019s 9% simply happened by chance).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In A/B tests, a null hypothesis typically states that the change (or changes you made on the page) have no effect on conversions.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">We reject the null hypothesis if the p-value is less than the significance level we set (more on this below). Rejecting the null hypothesis means our test shows evidence that there\u2019s a \u201cstatistically significant\u201d difference between the 7% and 9% conversion rates we saw earlier.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Having a \u201cstatistically significant\u201d result in our A/B test indicates that the change we made to the landing page probably had an impact on the conversion rate.</span></p>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-significance-level-and-p-value\">Significance Level and p-value</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">The significance level is the probability that your A/B test incorrectly rejects a null hypothesis that\u2019s actually true (i.e., the chance that you conclude there\u2019s an effect when there\u2019s really none). In other words, the significance level is the probability of getting a false positive result (or a Type 1 error).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">It\u2019s up to you how much significance level to use, but this is typically set to 5%. Having a 5% significance level means you\u2019re willing to accept a 5% chance of a false positive result in your A/B test.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">A related concept is the p-value. Statistics textbooks define The p-value as the probability that the result would be at least as extreme as those observed, assuming the null hypothesis was true.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">If you get confused by the \u201cassuming the null hypothesis was true\u201d portion, think of it as simply assuming you ran a test that\u2019s only made up of the control group (i.e., you made no variation).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Let\u2019s say that in our landing page split-test example, we got a p-value of 3.2% or 0.032. This means there\u2019s a 3.2% chance of getting at least a 9% conversion rate for the green-buttoned landing page (the variation group), assuming that the variation\u2019s conversion rate was the same as the control\u2019s 7% conversion rate.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Since we set the significance level at 5%, the p-value lies within the rejection threshold. This means it\u2019s very unlikely we got the 9% conversion rate assuming the null hypothesis is true. This is taken as evidence against the null hypothesis, and so we reject it.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In other words, the p-value simply tells us how surprising a given result is. If it\u2019s very surprising (i.e., p-value is less than the significance level), then it\u2019s most likely safe to reject the null hypothesis.</span></p>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-statistical-power\">Statistical Power</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Statistical power refers to the probability that your A/B test will correctly reject a false null hypothesis. In plain English, it\u2019s the chance that your test detects a specific effect when an effect actually exists.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">A low-power A/B test will be less likely to pick out an effect than a high-power test. The higher the statistical power, the lower the chance that your test makes a Type 2 error (failing to reject a false null hypothesis or false negative).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">According to ConversionXL, A/B tests follow an 80% power standard. To improve your test\u2019s statistical power, you need to increase the sample size, increase the effect size, or extend the test\u2019s duration.</span></p>\n\n\n\n<h3 class=\"wp-block-heading cb-heading-secondary\" id=\"h-effect-size\">Effect Size</h3>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In order for your A/B tests to be actionable and useful, you not only need to determine if a given variation has an effect, but you should also measure how much </span><i><span style=\"font-weight: 400;\">is</span></i><span style=\"font-weight: 400;\"> the effect. The significance level, p-value, and statistical power make up only the starting point. You also need to analyze the <a href=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC3444174/\">effect size</a>.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In our example earlier, the effect size is the absolute difference between the two group\u2019s conversion rates (2 percentage points). We may also express the effect size as units of standard deviation.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">It\u2019s important to estimate and/or compute the effect size in an A/B test. Estimating the effect size at the start of a test helps you determine the sample size and statistical power while reporting the test\u2019s post-experiment effect size allows you to make more informed decisions about the variations you\u2019re analyzing.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\">Related:&nbsp;<a href=\"https://mercurylabsmarketing.com/lead-generation/b2b-lead-generation-works-not-works/?utm_source=aaILnam\">B2B Lead Generation: What Works and What Doesn&#8217;t?</a></p>\n\n\n\n<h2 class=\"wp-block-heading cb-heading-primary\" id=\"h-confidence-intervals\">Confidence Intervals</h2>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">The 7% and 9% conversion rates from our earlier example are called point estimates (i.e., each of them corresponds to a single estimated number). But, since these values have only been estimated from samples, they may or may not coincide with the true conversion rates for each group.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">That\u2019s why you also need to build confidence intervals for your estimated conversion rates. Confidence intervals measure the reliability of an estimate by specifying the range of likely values where the true conversion rate will probably be found.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">For example, here\u2019s how we would most likely report a confidence interval for the variation\u2019s conversion rates: \u201cWe are 95% confident that the true conversion rate for the green-colored landing page is 9% +/- 2%.\u201d</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">In this example, we\u2019re saying that given the test results we have, our best estimate for the tweaked landing page\u2019s conversion rate is 9% and that we\u2019re 95% confident that the true conversion rate lies within 7% to 11%. The \u201c+/-2%\u201d value is called the margin of error.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Since we\u2019ve also made a point estimate of the control group\u2019s conversion rate, we need to construct a separate confidence interval for it. If we find, for example, that a 95% confidence interval for the control group\u2019s conversion rate overlaps with the other landing page\u2019s confidence interval, we may need to keep testing to arrive at a statistically valid result.</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Keep in mind that, in general, the larger the sample size, the narrower the confidence interval becomes (since more samples mean a more reliable estimate).</span></p>\n\n\n\n<p class=\"wp-block-paragraph\"><b>More Resources</b></p>\n\n\n\n<p class=\"wp-block-paragraph\"><span style=\"font-weight: 400;\">Here\u2019s a list of helpful resources and further readings on A/B testing statistics and inferential statistics in general:</span></p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https://conversionxl.com/blog/ab-testing-guide/\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"font-weight: 400;\">A/B Testing Mastery: From Beginner To Pro in a Blog Post</span></a><span style=\"font-weight: 400;\"> (ConversionXL)</span></li>\n\n\n\n<li><span style=\"font-weight: 400;\">Using Effect Size\u2014or Why the P Value Is Not Enough (NCBI)</span></li>\n\n\n\n<li><a href=\"https://www.incontext.indiana.edu/2008/nov-dec/4.asp\" target=\"_blank\" rel=\"noreferrer noopener\"><span style=\"font-weight: 400;\">The Search for Significance</span></a><span style=\"font-weight: 400;\">: A Crash Course in Statistical Significance (InContext)</span></li>\n</ul>\n","protected":false},"excerpt":{"rendered":"<p>Discover how A/B testing conversion works using data-driven visuals. Learn how to boost results with smart, strategic testing.</p>\n","protected":false},"author":9,"featured_media":26459,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_callbox_draft_pusher_loaded":"yes","footnotes":""},"categories":[178],"tags":[1966,1373],"class_list":["post-26457","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-growth-hacking","tag-ab-testing-statistics","tag-callbox-infographic"],"acf":{"cb_acf_post_contact_cta":"","cb_acf_post_contact_desc":"","contact_utm":"","contact_utm_button":"","cb_acf_post_offer_type":"ebook","cb_acf_post_offer":[24917],"cb_acf_guest_post":{"author_avatar":"","author_bio":""},"featured_case_study":null,"blog_faq":{"title":"","faq_list":null},"cb_acf_hero":{"h1_title":"","headline":"","description":"","image":"","video":"","video_header":false,"cta_link":"","cta_link_hs":"","cta":"BOOK A MEETING"},"has_contact_form":false,"dark_themed_form":false,"has_review_section":false,"top_level_page":false,"featured_services":{"section_title":"","section_description":"","featured_service_cards":{"card_1":{"title":"","description":"","button":null},"card_2":{"title":"","description":"","button":null},"card_3":{"title":"","description":"","button":null},"card_4":{"title":"","description":"","button":null}}},"leadgen_apac":false,"leadgen_au":false,"cb_asset":"","cb_web_property":"MAIN","cb_acf_relational":{"case_study":"","cs_industry_image":""},"cb_acf_testimonial":"","cb_regional_pages":null,"cb_acf_ctas":{"contact_form":"","banner_image":"","goto_form_button":"BOOK A MEETING"},"cb_acf_post_alt":{"alt_post_title":"","alt_post_image":"","alt_post_excerpt":"","alt_post_image_url":""},"form_redirect_url":"","cb_banner_announcement":null,"hubspot_form_guid":"6c1dda45-65a3-49f4-9de6-7bc157f6abbb"},"yoast_head":"","yoast_head_json":{"title":"Master A/B Testing Conversion with This Visual Guide","description":"Gain an intuitive understanding of key A/B testing statistics and math concepts with this simplified visual guide and in-depth blog post.","robots":{"index":"index","follow":"nofollow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https://mercurylabsmarketing.com/growth-hacking/math-behind-ab-testing-visual-guide/","og_locale":"en_US","og_type":"article","og_title":"The Math Behind A/B Testing: A (Simplified) Visual Guide - Mercury Labs","og_description":"Gain an intuitive understanding of key A/B testing statistics and math concepts with this simplified visual guide and in-depth blog post.","og_url":"https://mercurylabsmarketing.com/growth-hacking/math-behind-ab-testing-visual-guide/","og_site_name":"Mercury Labs Inc. - B2B Lead Generation Company","article_publisher":"https://www.facebook.com/Callbox/","article_author":"https://www.facebook.com/rbecca.matias","article_published_time":"2018-06-25T18:00:33+00:00","article_modified_time":"2026-03-27T16:08:10+00:00","og_image":[{"width":800,"height":352,"url":"https://mercurylabsmarketing.com/wp-content/uploads/2018/06/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide.jpg","type":"image/jpeg"}],"author":"Rebecca Matias","twitter_card":"summary_large_image","twitter_title":"The Math Behind A/B Testing: A (Simplified) Visual Guide - Mercury Labs","twitter_description":"Gain an intuitive understanding of key A/B testing statistics and math concepts with this simplified visual guide and in-depth blog post.","twitter_creator":"@https://www.twitter.com/becca_matias","twitter_site":"@callboxinc","twitter_misc":{"Written by":"Rebecca Matias","Est. reading time":"10 minutes"},"schema":{"@context":"https://schema.org","@graph":[{"@type":"Article","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#article","isPartOf":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/"},"author":{"name":"Rebecca Matias","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#/schema/person/016785973ca6dd662e4fb69361911775"},"headline":"The Math Behind A/B Testing: A (Simplified) Visual Guide [INFOGRAPHIC]","datePublished":"2018-06-25T18:00:33+00:00","dateModified":"2026-03-27T16:08:10+00:00","mainEntityOfPage":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/"},"wordCount":2322,"commentCount":0,"publisher":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#organization"},"image":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#primaryimage"},"thumbnailUrl":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2018/06/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide.jpg","keywords":["A/B Testing Statistics","Mercury Labs Marketing Infographics"],"articleSection":["Growth Hacking"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#respond"]}]},{"@type":"WebPage","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/","url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/","name":"Master A/B Testing Conversion with This Visual Guide","isPartOf":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#website"},"primaryImageOfPage":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#primaryimage"},"image":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#primaryimage"},"thumbnailUrl":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2018/06/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide.jpg","datePublished":"2018-06-25T18:00:33+00:00","dateModified":"2026-03-27T16:08:10+00:00","description":"Gain an intuitive understanding of key A/B testing statistics and math concepts with this simplified visual guide and in-depth blog post.","breadcrumb":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#primaryimage","url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2018/06/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide.jpg","contentUrl":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2018/06/The-Math-Behind-AB-Testing-A-Simplified-Visual-Guide.jpg","width":800,"height":352,"caption":"The Math Behind A/B Testing: A (Simplified) Visual Guide"},{"@type":"BreadcrumbList","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/growth-hacking/math-behind-ab-testing-visual-guide/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://mercurylabsmarketing.com/__callbox_upstream/cms/"},{"@type":"ListItem","position":2,"name":"The Math Behind A/B Testing: A (Simplified) Visual Guide [INFOGRAPHIC]"}]},{"@type":"WebSite","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#website","url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/","name":"Mercury Labs Inc. - B2B Lead Generation Company","description":"&lt;strong&gt;&lt;em&gt;B2B PR / Content&lt;/em&gt;&lt;/strong&gt;","publisher":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://mercurylabsmarketing.com/__callbox_upstream/cms/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#organization","name":"Mercury Labs Inc.","url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#/schema/logo/image/","url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2019/11/logo.png","contentUrl":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-content/uploads/2019/11/logo.png","width":107,"height":56,"caption":"Mercury Labs Inc."},"image":{"@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#/schema/logo/image/"},"sameAs":["https://www.facebook.com/Callbox/","https://x.com/callboxinc","https://linkedin.com/company/callbox-encino-california","https://www.pinterest.com/callboxinc/",""]},{"@type":"Person","@id":"https://mercurylabsmarketing.com/__callbox_upstream/cms/#/schema/person/016785973ca6dd662e4fb69361911775","name":"Rebecca Matias","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://secure.gravatar.com/avatar/7a0f51f86f8b8079ad9a5bda0883cefa3069e954d30ba41117bb8968a3b861f5?s=96&r=g","url":"https://secure.gravatar.com/avatar/7a0f51f86f8b8079ad9a5bda0883cefa3069e954d30ba41117bb8968a3b861f5?s=96&r=g","contentUrl":"https://secure.gravatar.com/avatar/7a0f51f86f8b8079ad9a5bda0883cefa3069e954d30ba41117bb8968a3b861f5?s=96&r=g","caption":"Rebecca Matias"},"description":"Rebecca Matias is Mercury Labs's COO with 18 years of experience scaling B2B pipeline through data-driven outbound marketing, lead generation, and sales development.","sameAs":["https://mercurylabsmarketing.com","https://www.facebook.com/rbecca.matias","https://www.linkedin.com/in/rebecca-matias-callbox-usa","https://x.com/https://www.twitter.com/becca_matias"],"url":"https://mercurylabsmarketing.com/__callbox_upstream/cms/author/rebecca/"}]}},"_links":{"self":[{"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/posts/26457","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/posts"}],"about":[{"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/types/post"}],"author":[{"embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/users/9"}],"replies":[{"embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/comments?post=26457"}],"version-history":[{"count":27,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/posts/26457/revisions"}],"predecessor-version":[{"id":120339,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/posts/26457/revisions/120339"}],"acf:post":[{"embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/pages/24917"}],"wp:featuredmedia":[{"embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/media/26459"}],"wp:attachment":[{"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/media?parent=26457"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/categories?post=26457"},{"taxonomy":"post_tag","embeddable":true,"href":"https://mercurylabsmarketing.com/__callbox_upstream/cms/wp-json/wp/v2/tags?post=26457"}],"curies":[{"name":"wp","href":"https://api.w.org/{rel}","templated":true}]}}