How Many Creatives to Test at Once
The most common mistake is launching ten creatives with a $50 budget for each and hoping the algorithm finds the winner. It doesn’t work that way. With a small number of impressions, the difference in CTR and CVR between variants almost always falls into the noise. You get a ranking where the first and second places would swap on a rerun.
The rule we use at the start of a campaign: test no more than five creatives simultaneously. This number isn’t pulled out of thin air—it allows allocating enough impressions to each variant for an initial comparison without spreading the budget too thin. If the daily campaign budget is $500, five creatives get $100 a day each. That’s enough to see a difference in CTR and CPM in 2–3 days. Drawing conclusions about CVR and CPI requires more time—usually 4–7 days depending on traffic volume.
If your budget is under $200 a day, cut the number of variants down to three. It’s better to get a reliable signal on three creatives than an unreliable one on five.
What to Compare at Each Stage
Creative testing isn’t a single test, but a sequence of three. Each answers its own question.
First test — clickability. Shows whether the creative grabs attention in the feed. Metric: CTR. Threshold: 30–40% above the account or platform average. We turn off creatives that don’t pass this threshold—there’s no point paying for impressions that don’t convert into clicks.
Second test — conversion. Shows if the creative drives the viewer to the target action. Metric: post-click CVR. Here, it’s important to look not just at installs, but at in-app events—registration, first episode, purchase. A creative with high CTR and low CVR often means the promise in the creative doesn’t match what the user sees after installing.
Third test — scalability. Shows how the creative behaves when budget and audience increase. Metric: CAC at 3x and 5x of the starting budget. A creative that cost $2 per install at $100 a day might drop to $6 at $500. This doesn’t mean it’s bad—it means its scalability is limited.
How to Distribute Budget Among Variants
There are two approaches, and both are viable, but they work in different situations.
Equal distribution. Each creative gets an equal share of the budget. Suitable when all variants are made to the same brief and you don’t know which will work. This is the cleanest way to compare because the algorithm doesn’t have time to reallocate impressions in favor of one variant.
Dynamic distribution. The platform algorithm reallocates the budget itself in favor of the creative with the best result for the chosen metric. Suitable for the second and third tests when you’ve already weeded out obvious losers and want to find the leader faster. But in the very first test, dynamic distribution is risky: the algorithm might bet too early on a variant that randomly got a couple of cheap installs in the first hour.
Our approach: first test—equal distribution, manual turning off of obvious losers after 48 hours. Second test—dynamic distribution among the remaining 2–3 variants. Third test—manual budget management of the leader with a gradual increase.
When to Kill a Creative
Turning off creatives is hard, especially if someone on the team made them. But keeping a weak creative in an active campaign is a direct loss: it steals impressions from stronger variants and ruins the average CAC.
Signals to turn it off:
- CTR below 60% of the leader after 48 hours and 5,000 impressions.
- CPM 25% above the campaign average and not dropping.
- CVR after 100 clicks is zero or half that of the leader.
- Impression frequency for this creative is above 3, and CTR drops from impression to impression.
If a creative fails at least one of these thresholds, turn it off. Don’t wait for it to “ramp up.” Creatives that don’t show results in the first 48 hours with sufficient impression volume almost never become leaders at scale.
The Early Conclusion Trap
The most expensive mistake is drawing conclusions from day-one data. On the first day after launch, the platform algorithm is just gathering audience data. CPM is unstable, impressions might go to the wrong audience. A creative that looks terrible in the first 12 hours might become a leader by day three.
The minimum timeframe for an initial conclusion is 48 hours and at least 3,000 impressions per creative. If impressions are lower, wait longer. If your budget doesn’t allow 3,000 impressions in 48 hours, you’re testing too many variants or on too narrow an audience.
How to Read the Difference Between CTR and CVR
A creative with high CTR and low CVR isn’t a bad creative. It’s a creative with a false promise. It attracts clicks, but viewers don’t find what they expected and leave. The problem isn’t the visual, but the disconnect between the creative and the product.
Solution: check what exactly you show in the first seconds after a click. If the creative promises a comedic moment, but the user lands on a registration screen—the disconnect is obvious. If the creative features a specific character, but they aren’t on the landing screen—same effect.
A creative with low CTR and high CVR is usually a niche creative. It attracts few people, but converts those who click well. Such creatives are useful for precise targeting but aren’t suitable for scaling. Keep them in the campaign as auxiliary, but don’t rely on them when increasing the budget.
The ideal creative is average CTR and high CVR. It doesn’t necessarily win on clickability, but it brings cheap installs because the viewers who click follow through.
How Many Creatives You Need in Constant Rotation
After testing is complete and 1–2 leaders are chosen, the work isn’t over. Creatives suffer from fatigue—this is a fact you can’t change. Fatigue manifests in a gradual increase in CPM and a drop in CTR as impression frequency rises.
A healthy rotation for a medium-scale mobile campaign is 3–5 active creatives simultaneously. Not one, because the algorithm works better with a choice. Not ten, because the budget gets spread thin. At the same time, you should have 2–3 ready creatives in reserve to launch when the active ones start showing signs of fatigue.
Signs of fatigue: CTR drops by 15–20% from the peak at the same impression volume, CPM rises, frequency exceeds 4. When two of the three appear—launch the next creative from the reserve and gradually lower the budget on the fatiguing one.
What to Do With Creatives That Failed the Test
Don’t delete them. A creative that didn’t work for one audience might work for another. A creative that didn’t show results in one geo sometimes works in a neighboring market with a different cultural context.
We store all creatives with notes: test date, audience, geo, result, reason for turning off. Once a quarter, we review the archive and sometimes find candidates for a retest in a new market or a new placement format. About 15% of creatives that failed the first test get a second chance—and about a third of them work the second time around.
Creative Test Launch Checklist
- No more than five creatives in one test, three if the budget is under $200 a day
- Equal budget distribution on the first test, dynamic on the second
- Minimum 48 hours and 3,000 impressions per variant before initial conclusions
- Turn off creatives with CTR below 60% of the leader after 48 hours
- Check creative-to-landing-screen alignment when there’s a CTR and CVR gap
- Keep 3–5 active creatives and 2–3 in reserve for replacement
- Archive failed creatives with metadata for retesting in other markets
FAQ
How many days should a creative test run?
At least 48 hours for CTR conclusions and 4–7 days for CVR and CPI conclusions. A shorter period doesn’t provide a statistically reliable signal.
What if all five creatives show poor results?
Check the campaign settings, not the creatives: audience, placement, landing screen. If all variants are equally bad, the problem usually isn’t the visual.
Can I test creatives on a narrow audience?
Yes, but the results won’t scale. A narrow audience gives a cheap signal to check the visual, but the CAC will be different when expanded.
Should I change creatives simultaneously on all platforms?
No. Test on one platform, then move the winner to the others. Simultaneous replacement everywhere deprives you of a comparison point.
What is a good CTR for mobile video?
It depends on the geo, format, and platform. Benchmark: 1–2% for video inserts in developed markets, 2–4% for rewarded formats. Compare not with an absolute, but with your account average.
