with the skill
The cleanest test is to **change only the psychological message around the CTA**, while keeping the button, offer, and page identical. That helps you learn whether scarcity or social proof drives more action—not whether one button label is clearer.
## 1. Define the test
**Research question:** Among visitors to this page, does truthful scarcity or relevant social proof generate more completed conversions?
| | A: Scarcity | B: Social proof |
|---|---|---|
| Psychological mechanism | A genuine limit gives people a reason to act now. | Evidence of others’ experience reduces uncertainty. |
| Message above CTA | “Only **[verified number]** onboarding slots remain for **[month]**.” | “Join **[verified number] [relevant customer type]** using **[product]**.” |
| Button—identical in both | “Book a demo” | “Book a demo” |
These are templates, not claims to publish without evidence. For social proof, a specific, permissioned customer quote may work better than a customer count if trust is the main barrier.
**Important:** If you don’t have a genuine capacity limit, deadline, or inventory constraint, don’t manufacture scarcity. Use another comparison instead.
## 2. Keep everything else constant
- Randomly assign eligible visitors **50/50**, concurrently.
- Keep visitors in the same variant on repeat visits.
Show the rest (37 more lines)
- Hold layout, button copy, color, placement, pricing, offer, and traffic targeting constant.
- Give both messages comparable visual prominence and similar length.
- Exclude employees, bots, and test traffic using rules set before launch.
For B2B sites where colleagues may influence the same purchase, consider account-level assignment if you can implement it reliably.
## 3. Measure the outcome—not just the click
**Primary metric:** Completed target actions per assigned visitor—for example, completed demo bookings, signups, or purchases.
**Secondary metric:** CTA click-through rate.
**Guardrails:** Lead quality, cancellations, refunds, or early retention, depending on the action.
Scarcity could increase clicks while attracting rushed, lower-quality conversions. That’s not necessarily a win.
## 4. Set the sample size and stopping rule before launch
You’ll need:
- Current conversion rate.
- Eligible visitors per week.
- The smallest lift worth acting on—your **minimum detectable effect**.
A conventional starting point is **80% power and a two-sided 5% significance level**. For illustration, detecting a rise from **5% to 6%**—a 20% relative lift—requires roughly **8,200 visitors per variant** under a standard two-proportion calculation.
Run until you reach the planned sample size **and cover at least two full weekly cycles**. Don’t stop early just because a standard significance dashboard briefly shows a winner; use a valid sequential-testing method if you need continuous stopping.
## 5. Decide what the result means
- **Meaningful conversion lift, no guardrail harm:** Roll out the winner.
- **More clicks, no completed-conversion lift:** Investigate post-click friction; don’t call it a business win.
- **Inconclusive:** Neither variant is proven better. Consider whether more data is worthwhile.
- **Different results by segment:** Treat unplanned segment findings as hypotheses for a follow-up test.
**Interpretation limit:** This tests *these two messages with this audience and offer*, not whether scarcity universally beats social proof. Without a neutral control, it also won’t tell you whether either beats your current CTA experience.
**Send me your current CTA and surrounding copy, the action you want visitors to complete, weekly traffic, baseline conversion rate, and the truthful scarcity/social-proof evidence you have.** I can turn that into two launch-ready variants and a sample-size plan.