YouTube Thumbnail A/B Test Sample Size Calculator
Estimate the impressions a conventional CTR experiment may need to detect a chosen lift. Set your baseline, confidence and power to plan a test.
0
estimated impressions per variant
This is a conventional statistical planning estimate for CTR. It does not reproduce YouTube Test & Compare, its traffic allocation, stopping rules or winner selection. YouTube says its native tool evaluates thumbnail performance using watch time share, not CTR alone. For three variants, this estimate does not apply a multiple-comparison adjustment.
Small CTR changes need more evidence
A difference can look convincing before there is enough traffic to distinguish a real change from random variation.
The calculator estimates the impressions needed per variant for a conventional two-proportion comparison. Smaller detectable lifts, higher confidence and higher power increase the sample size. The result uses a normal approximation and assumes independent groups with a fixed sample.
How this differs from YouTube Test & Compare
YouTube's native Test & Compare is its own experiment, with its own allocation and winner-selection method. YouTube says thumbnail performance is assessed using watch time share. This calculator plans a conventional CTR comparison and does not reproduce YouTube's internal methodology or identify the winner of a YouTube experiment.
Confidence and power
Confidence sets the two-sided significance level used by the planning formula. Power is the chance of detecting the selected difference if it is present, under the model assumptions. Neither setting accounts for every source of bias or change in a real audience.
When testing three variants, this calculator multiplies the per-variant estimate to show a rough total. It does not adjust significance for multiple comparisons, so treat the number as an initial planning guide.
Thumbnail A/B testing FAQ
How many impressions do I need for a thumbnail test?
There is no universal number. The required sample depends on baseline CTR, the smallest lift you want to detect, confidence, power and the assumptions of the model. YouTube's own Test & Compare may use a different methodology.
Should I optimize a YouTube thumbnail for CTR only?
CTR alone does not show whether viewers spend time watching after they click. YouTube says its native Test & Compare evaluates thumbnail performance using watch time share.
What does a 10% lift mean?
It is a relative increase. A 10% lift from 4% CTR is 4.4%, which is an increase of 0.4 percentage points.
Why do small improvements need more impressions?
When two rates are close, more observations are needed to distinguish their difference from ordinary random variation at the chosen confidence and power.
Can I compare three thumbnails with this calculator?
Yes. Choose three variants to estimate a per-variant sample and rough total. The estimate does not include a multiple-comparison adjustment and does not reproduce YouTube's native experiment.