Experiment Results Calculator: is the difference real or chance?
Compare two ads, pages or experiment arms and see if the difference is likely real or chance.
- Free to use
- Runs in your browser
- No sign-up
Your result appears here
Enter the numbers for both versions, or try the example: two ads for a gym in Pune.
Want an expert to check this result?
Send it to us in one click. We read it and reply within one business day with what we would fix first.
Embed this tool on your websiteFree. Paste one code; it runs in your visitors' browsers.
Copy this code into a page of your site. The tool sets no cookies and sends nothing your visitors type to us. Change height="760" if it needs more room.
About the Experiment Results Calculator
Enter the impressions, clicks, conversions, cost and conversion value of two versions: two ads, two landing pages, a Google Ads experiment's original and trial, or before and after a change. Pick the metric (CTR, conversion rate, cost per conversion or ROAS) and a confidence level of 90, 95 or 99%. You get each version's figure, the change and its range, a plain verdict, how much more data you need and how Google Ads reports its own experiments. It runs in your browser and nothing is uploaded.
How it works
-
Step 1
Choose what you are comparing, then enter the numbers for each version for the same dates from Google Ads.
-
Step 2
Pick the metric that matters for the test and the confidence level; 95% is the usual choice.
-
Step 3
Read the verdict and the range, check how much more data you need, and copy the summary for your notes or your client.
How the calculator decides
For CTR and conversion rate it uses the two-proportion z-test, the textbook test for comparing two rates, with a range for the difference at the confidence level you pick. Cost per conversion and ROAS have no exact test from totals, so it uses an approximation: conversions are treated as counts that vary by chance, while spend and value per conversion are treated as fixed. The page shows the cost per click and conversion rate behind the change separately.
How Google Ads reports experiments
Google's experiment scorecard shows each metric's change with a confidence interval, and marks results that are statistically significant. Google uses jackknife resampling on 20 buckets per version and a two-tailed test at 95% confidence. Search experiments can split traffic by cookie or by search, and Google says a 50% split gives the best comparison.
What this calculator cannot do
It works on totals, so it cannot repeat Google's method or see day-to-day swings; for a Google Ads experiment, trust the scorecard. Two ads in one ad group, or before and after a change, are not split at random, so other things can cause a difference. Looking at results every day and stopping at the first good one makes false wins more likely, so decide the length of the test first.
Frequently asked questions
How do I know if my Google Ads test result is statistically significant?
In a Google Ads experiment, the scorecard shows each metric's change with a confidence interval, and Google marks results that are statistically significant. For two ads or two pages outside an experiment, enter both sets of numbers here: if the range for the change does not cross zero at your confidence level, the difference is likely real.
What confidence level should I use?
95% is a common choice: a gap this big would show up by chance less than 5 times in 100 when there is no real difference. Our suggestion is 90% for a quick, low-risk decision and 99% when a change is costly to undo. Google uses a two-tailed test at 95% confidence for its own experiments.
How long should a Google Ads experiment run?
Decide before you start. The first end date of an experiment must be 2 to 12 weeks after the start, and you can extend it. For Shopping and Performance Max experiments, Google suggests at least 4 to 6 weeks, longer with a long conversion delay, and drops the first 7 days. This calculator estimates how much more data you need.
Should I use a cookie-based or search-based split?
For Search custom experiments, Google recommends the cookie-based split, where each person sees only one version. A search-based split assigns each search again, so one person can see both versions, but it may reach a significant result faster. Google says a 50% split gives the best comparison.
Why does the calculator say "not enough data yet"?
The difference you see could still be chance at the confidence level you picked. Google lists the same reasons for its own experiments: too little traffic, a split that is too small, or a change that made no real difference. The calculator shows how many more clicks or conversions you need to be sure.
Related reading
Want help with your Google Ads?
Tell us about your account and what your ads should bring in. We reply within one business day.
Talk to us Our service: Google Ads managementMore Google Ads tools
All Google Ads tools- Ad Metrics Calculator ROAS, ROMI, CPA, CPC, CPM, CTR and CPV from your ads report, with break-even ROAS and GST.
- Google Ads Report Builder Turn a Campaigns export into a clear monthly report with changes, charts and notes, ready as a PDF.
- True ROAS Calculator See what your Google Ads ROAS is worth after COD, RTO, returns, shipping, fees and GST.
- Budget Pacing Calculator See if this month's spend is on track and the daily budget that lands on your monthly target.
Want us to fix what the tool found?
Tell us about your business and what you want to grow. We reply within one business day with next steps, not a sales script.
- A reply within one business day
- A quick look at your site and search visibility before we talk
- Your details are only used to answer your enquiry