with the skill
# A/B Test Plan: Scarcity vs. Social Proof CTA
I haven't seen your product context, so this is a framework you can adapt. The questions at the end will let me tailor it.
## 1. Hypothesis
> For [audience] on [page/placement], a CTA using **social proof** will produce a higher [click / signup / purchase] rate than one using **scarcity**, because [reason tied to your buyer].
Which one tends to win depends on the situation:
| | Scarcity / urgency | Social proof |
|---|---|---|
| Works best when | Buyers are already interested and procrastinating | Buyers are unsure, new to the brand, or the product is unfamiliar |
| Psychology | Loss aversion, regret aversion | Bandwagon effect, uncertainty reduction |
| Risk | Erodes trust if it feels fake | Weak if the numbers are small or generic |
Use this to pick a prediction before you test. For example, cold traffic to a new brand probably favors social proof, and warm leads at the decision stage probably favor scarcity.
## 2. Variants
Show the rest (53 more lines)
I recommend **three arms**, not two. A head-to-head only tells you which is better, not whether either beats doing nothing.
- **Control:** your current CTA
- **Variant A (scarcity):** *"Get started, only 12 spots left this month"*
- **Variant B (social proof):** *"Join 4,200+ teams already using [Product]"*
Rules to keep the test clean:
- **Change one variable.** Keep the button color, placement, layout, and base CTA verb the same. Only the persuasion element changes.
- **Match the length and prominence** of both messages. Otherwise you're testing visual weight, not psychology.
- **Both claims must be true.** Scarcity must reflect real limits (actual inventory, seats, or a real deadline). Social proof must use real, verifiable numbers. Fake versions can win a test and still damage trust, increase refunds, and create legal exposure.
## 3. Metrics
- **Primary:** one metric tied to the CTA's purpose (click-to-signup, trial start, or purchase). Decide this before launching.
- **Secondary:** CTA click-through rate, bounce rate.
- **Guardrails:** refund or cancellation rate, trial-to-paid conversion, and 30-day retention. Scarcity often lifts top-of-funnel conversion but attracts lower-intent buyers who churn, so a "win" on clicks can be a loss on revenue.
## 4. Sample size and duration
A rough rule per arm, at 95% confidence and 80% power:
**n ≈ 16 × p(1−p) / δ²** (p = baseline conversion, δ = absolute lift you want to detect)
Example: with a 3% baseline and a goal of detecting a 20% relative lift (3.0% → 3.6%), you need about **13,000 visitors per arm**, so roughly 39,000 total for three arms.
- Divide that by your daily traffic to get the duration.
- Run for **full weeks, at least 1–2 cycles**, to smooth out day-of-week effects.
- **Don't stop early** when one variant "looks ahead." Peeking inflates false positives.
- If traffic is too low for a 20% lift, test a bolder change or accept that you can only detect larger effects.
## 5. Common pitfalls
- **Novelty effect:** early lifts often fade, so check that the trend holds in week 2.
- **Segment mixing:** the winner may differ for new vs. returning visitors or by traffic source. Plan those cuts in advance, but treat them as hypotheses, not conclusions.
- **Urgency fatigue:** scarcity can work in a short test and wear out over time. Social proof usually ages better.
- **Overlapping tests:** don't run another test on the same page at the same time.
- **Randomization errors:** confirm the split is truly random and that users see a consistent variant on return visits.
## 6. Decision rules (set these before launch)
- Winner = statistically significant lift on the primary metric **and** no degradation on guardrails.
- If neither beats control, you have learned that persuasion copy isn't your bottleneck. Consider the Theory of Constraints: the problem may be traffic quality, offer, or page friction rather than CTA wording.
- If both win, a follow-up test could combine them (e.g., "Join 4,200+ teams, 12 spots left"), but only once you know each one's individual effect.
## What I need from you
1. What is the CTA for (signup, purchase, demo request, trial), and where does it sit?
2. What's your current conversion rate and weekly traffic to that page?
3. Who is the audience: cold traffic, warm leads, or existing customers?
4. Do you have real scarcity (limited seats, a deadline, inventory) and real social proof numbers to use?
5. What testing tool are you using (Optimizely, VWO, GA4 experiments, in-house)?
With these I can write the exact copy variants, calculate your sample size and runtime, and sketch the results-analysis template.