This A/B test hypothesis generator turns supplied funnel data and customer observations into a prioritized experiment backlog. It helps ecommerce teams specify what to change, who the change is for and which metric would indicate progress, so ideas such as clearer shipping information become testable proposals.
Give the Skill stage counts, the current conversion rate, relevant customer segments and implementation constraints. It looks for points where shoppers drop out, then connects each proposed change to an expected behavior and a reason grounded in the supplied evidence.
The result is a ranked set of hypotheses and variant ideas for the team to review. Sample-size calculations, experiment setup and the eventual decision about a winner are separate steps.
A useful hypothesis links a specific change to a measurable outcome and a plausible mechanism. The structured format makes it easier to challenge vague ideas such as “make the page more persuasive” before spending design or engineering time.
Compare ideas using the evidence available, likely effort and business constraints. It suits checkout-friction reviews, product-page tests and campaign landing pages where several competing improvements need an explicit order.
Funnel counts, segment definitions, baseline conversion, traffic, business constraints, and implementation effort.
A model chat is enough to draft from these instructions. Real system access or file changes need separately configured tools.
There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.
Use these instructions in a chat, or ask Codex / Claude Code to save them as a personal Skill for repeated tasks.
Set up a personal reusable instruction template for A/B Test Hypothesis Generator in the app I am using.
1. Read the “Collected instruction template” section at https://ecomagenttools.com/skills/ab-test-hypothesis-generator. Use only that complete instruction text, not the page’s reviews, FAQ or installation explanation. If you cannot retrieve the complete text, ask me to paste it; do not guess it or download a same-named third-party package.
2. Show which assumptions, placeholders and access requirements I need to supply. Preserve the collected task instructions. This is a locally adapted template, not an original-author package.
3. If you are Codex or Claude Code with local file access, create a personal Skill named ab-test-hypothesis-generator-local under ~/.agents/skills/ab-test-hypothesis-generator-local/SKILL.md for Codex or ~/.claude/skills/ab-test-hypothesis-generator-local/SKILL.md for Claude Code. Add valid name and description frontmatter if absent; keep the collected instructions in the body and identify the local adaptation. Do not overwrite an existing Skill. If you are in a chat-only app, use these instructions in the current conversation instead and do not claim a local installation.
4. Return the saved path or current-chat setup result and a starter request for this task. List any required tools or data that remain unconfigured. Do not claim account/API access or a completed business workflow just because the template was saved.Network-collected; no confirmed author source. The collected text was reviewed, but it has not been linked to a verified original publication. This does not establish that no official package exists; the examples here use only the collected text.
You are a CRO (Conversion Rate Optimization) strategist for ecommerce stores. When given store analytics data, you will:
1. IDENTIFY bottlenecks:
- Pages with highest exit rates
- Steps with largest funnel drop-offs
- Segments with lowest conversion rates
- Devices with poor performance
2. GENERATE hypotheses (format: If [change], then [expected result], because [reason]):
- Quick wins (implement in <1 day)
- Medium effort (1-3 days)
- Major tests (1-2 weeks)
3. For each hypothesis, provide:
- Current metric baseline
- Expected improvement range
- Confidence level (Low/Medium/High)
- Sample size needed for statistical significance
- Test duration recommendation
- Revenue impact estimate
4. PRIORITIZE by: Expected Revenue Impact × Implementation Ease
5. SUGGEST test variants:
- Control (current version)
- Variant A (minimal change)
- Variant B (bold change)
- Variant C (radical change — optional)
Your recommendations should be specific, data-backed, and actionable. Never suggest changes without estimated impact.Starter prompts for the main use cases—copy and use them directly.
Compare funnel stages and devices, reporting denominators and data gaps. Use for: Find a funnel bottleneck. Prepare: Funnel counts, segment definitions, baseline conversion, traffic, business constraints, and implementation effort.
Use the collected instructions above. Task: Compare funnel stages and devices, reporting denominators and data gaps.. Inputs: [INPUT]. Constraints: [CONSTRAINTS]. Check missing information first. Produce a reviewable result with evidence, assumptions, and next steps. There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.[INPUT][CONSTRAINTS]Create If/Then/Because hypotheses with control and minimal-change variants. Use for: Write test hypotheses. Prepare: Funnel counts, segment definitions, baseline conversion, traffic, business constraints, and implementation effort.
Use the collected instructions above. Task: Create If/Then/Because hypotheses with control and minimal-change variants.. Inputs: [INPUT]. Constraints: [CONSTRAINTS]. Check missing information first. Produce a reviewable result with evidence, assumptions, and next steps. There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.[INPUT][CONSTRAINTS]Rank hypotheses by evidence, potential impact, and effort; separate assumptions from measured effects. Use for: Prioritize a test backlog. Prepare: Funnel counts, segment definitions, baseline conversion, traffic, business constraints, and implementation effort.
Use the collected instructions above. Task: Rank hypotheses by evidence, potential impact, and effort; separate assumptions from measured effects.. Inputs: [INPUT]. Constraints: [CONSTRAINTS]. Check missing information first. Produce a reviewable result with evidence, assumptions, and next steps. There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.[INPUT][CONSTRAINTS]Use two weeks of orders, sessions, device performance and funnel data. The current week has 4,800 sessions and 120 orders; distinguish traffic growth from conversion improvement before choosing changes.
Case input files: business-inputs.zip
Use the ab-test-hypothesis-generator Skill.
Our order count rose but the team disagrees about whether the product page improved. Recalculate the funnel and propose three prioritized experiments around the purchase button, shipping information and size imagery. For each, give the customer problem, control and variant, success metric, smallest useful change, sample size and likely duration at our traffic level. Recommend what we should do first this week.
The attached business materials are:
- inputs/brief.json
Deliver a complete result.md report that the operations team can read, plus any finished files needed for this task. Keep supporting calculations and evidence with the deliverables. Write the report in English; use the requested target language for customer-facing copy. The business datasets are fictional examples unless the brief explicitly names a live public website.The primary file produced by this prompt is shown below in its original language.
Start with the purchase button this week. Put the existing button immediately below price and above care information, and test that placement against the current page. Order growth alone does not show that the product page improved: orders and traffic both rose 20%, while conversion stayed at 2.50%.
The supplied analytics and interface descriptions are synthetic; all experiment lifts and revenue effects below are planning assumptions, not measured results. No live product page or historical product-page funnel was supplied.
| Metric | Prior week | Current week | Change |
|---|---|---|---|
| Sessions | 4,000 | 4,800 | +20% |
| Orders | 100 | 120 | +20% |
| Session-to-order conversion | 2.50% | 2.50% | 0 percentage points |
| Gross product revenue | $2,400 | $2,880 | +20% |
| Average order value | $24 | $24 | Unchanged |
| Gross revenue per session | $0.60 | $0.60 | Unchanged |
| Refund amount / gross revenue | 5% | 5% | Unchanged |
| Ad spend per order | $3 | $3 | Unchanged |
| Modeled contribution after ads* | $780 | $936 | +20% |
Gross product revenue minus refunds, $8 unit cost per order, $4 merchant shipping cost per order and supplied ad spend. This assumes one unit per order and excludes collected shipping fees, taxes, payment fees, overhead and any returned-stock recovery. It is not accounting profit. Refund amounts may relate to earlier orders; the 5% ratio is not a cohort return rate.
At the prior 2.50% conversion rate, 4,800 sessions produce exactly 120 orders. Increased traffic fully explains the observed order increase arithmetically. This does not establish what caused traffic growth or rule out offsetting changes within devices. A prior product funnel and randomized comparison are missing, so the team cannot establish a product-page improvement from these totals.
| Step | Entered | Progressed | Step conversion | Lost | Step drop-off |
|---|---|---|---|---|---|
| Session → product | 4,800 | 3,600 | 75.00% | 1,200 | 25.00% |
| Product → cart | 3,600 | 720 | 20.00% | 2,880 | 80.00% |
| Cart → checkout | 720 | 360 | 50.00% | 360 | 50.00% |
| Checkout → order | 360 | 120 | 33.33% | 240 | 66.67% |
Product-session-to-order conversion is 120 / 3,600 = 3.33%. Product → cart has both the largest absolute loss and the highest proportional loss. Checkout also deserves attention: two-thirds of checkout sessions do not become orders, and the supplied interface description reports a long single-column form. That is a separate investigation, not a change to bundle into these tests.
| Page | Views | Exits | Exit rate |
|---|---|---|---|
| Product | 4,200 | 2,100 | 50% |
| Cart | 900 | 270 | 30% |
| Checkout | 450 | 180 | 40% |
Product has the highest reported exit rate. Page views include repeat views; these exit rates are not funnel drop-off rates and should not be substituted for them.
| Device | Sessions | Orders | Conversion |
|---|---|---|---|
| Mobile | 3,600 | 60 | 1.67% |
| Desktop | 1,200 | 60 | 5.00% |
Mobile supplies 75% of sessions but only 50% of orders; its conversion is one-third of desktop's. This makes mobile usability the first diagnostic priority, but traffic quality or customer mix could also explain the gap. Device-level product sessions and cart progression were not supplied.
The primary success metric for all three is completed purchases / eligible product-page sessions, baseline 3.33%. Use add-to-cart and checkout progression as diagnostic metrics, not substitutes for purchase success. The common denominator allows revenue and test duration to be compared consistently.
| Rank | Experiment | Assumed purchase lift | Full-rollout added orders/week | Full-rollout added gross revenue/week | Added contribution/week* | Ease (1–5) | Priority score |
|---|---|---|---|---|---|---|---|
| 1 | Purchase-button placement | 15–25% relative | 18–30 | $432–$720 | $194–$324 | 5 | 2,880 |
| 2 | Shipping information | 10–20% relative | 12–24 | $288–$576 | $130–$259 | 4 | 1,728 |
| 3 | Size imagery | 5–15% relative | 6–18 | $144–$432 | $65–$194 | 2 | 576 |
Priority score = midpoint weekly gross revenue impact × implementation ease. Ease is a planning judgment, with 5 easiest. These are positive planning scenarios; actual impact could be zero or negative. Confidence in every numeric lift is Low, since the brief contains no experiment or customer research evidence. The button has the strongest directional rationale because the reported mobile placement and funnel loss align.
Incremental contribution assumes $24 × 95% − $8 − $4 = $10.80 per additional order, with ad spend held constant and the same exclusions as above. Revenue scenarios assume the existing 120 weekly orders all pass through the reported product funnel and that traffic, AOV and refund ratio stay constant. During a 50/50 test, the expected incremental store impact is about half the full-rollout estimate. The experiments overlap and their impacts must not be added together.
Customer problem: On mobile, the buy button sits below care copy. A shopper ready to buy may not immediately see how to proceed.
Hypothesis: If the existing purchase button appears immediately below price and above care copy, then more product sessions will become purchases, because ready-to-buy shoppers can find the next action sooner.
The all-device lift assumption is ambitious if only mobile presentation changes. Report mobile results separately, but do not claim a mobile-specific winner from this pooled sample plan. Mobile-only sizing requires mobile product-session volume and its purchase rate.
Customer problem: The cart first reveals shipping charges. Shoppers cannot assess cost and timing before committing to the next step.
Hypothesis: If the product page states shipping price, eligibility and timing beside the purchase action, then more product sessions will become purchases, because shoppers can judge the delivered offer before entering the cart.
Control: Shipping charge first revealed in cart.
Variant A — smallest useful change: Add a compact text block directly below the purchase button without changing rates or delivery promises:
Domestic shipping: $5 on orders under $50; free on orders of $50 or more. Dispatch within 2 business days. Delivery takes 3–5 business days after dispatch. No international delivery.
Confirm that the stated delivery window begins after dispatch before publishing; the supplied policy lists dispatch and delivery separately but does not explicitly define that timing relationship. If unconfirmed, use the literal policy wording: “Domestic dispatch: 2 business days. Delivery: 3–5 business days.” Keep the remaining shipping text above unchanged.
Variant B — bolder alternative: An expandable “Shipping & delivery” panel beside the action, displaying the current cart's actual shipping charge and the amount remaining to reach $50. Example for a one-mug cart: “Shipping: $5. Add $26 to reach free shipping.” Recalculate from actual cart value; do not change the threshold.
Baseline and target: Purchase rate 3.33%, targeting 3.67–4.00%. Cart-to-checkout is 50%, a secondary diagnostic. Earlier disclosure may reduce cart additions while improving purchase quality, so judge purchases first. Monitor gross revenue and contribution per exposed session.
Sample and duration: At a 20% relative lift, 12,475 sessions per arm; 24,950 total; about 6.93 weeks, budget 7 full weeks. Expected full-rollout revenue impact: +$288–$576/week. Lift confidence: Low.
Customer problem: One studio image gives little context for the mug's physical scale, potentially leaving shoppers unsure whether it fits their intended use.
Hypothesis: If the gallery shows verified dimensions and an accurate scale reference, then more product sessions will become purchases, because shoppers can better judge the mug's size.
The physical dimension mapping and scale assets are not supplied. These are completed variant specifications and copy drafts; an accurate customer-facing image cannot be finished from the brief alone.
Sample sizes use a two-sided two-proportion normal approximation, 5% significance, 80% power and 50/50 allocation. Each calculation compares control against Variant A only, using the upper end of that experiment's assumed lift range as the minimum effect the test is powered to detect. Smaller lifts require more traffic; a nonsignificant result at these sample sizes would not rule out the lower end of the range. Variant B is an alternative concept, not covered by these sample sizes or impact estimates; size it separately before use.
Duration assumes 3,600 eligible product sessions per week, or 1,800 per arm, no traffic ramp and no exclusions. These are three sequential planning estimates, not a promise that all three can finish in parallel. Repeated sessions from the same shopper are not independent: assign visitors persistently, analyze at the randomization unit and recalculate sample needs from deduplicated eligible visitors before launch. Session-based figures here are provisional because unique-visitor counts are absent.
Use a consistent purchase-attribution window, fixed before launch; allow that window to mature after exposure ends. Verify that orders can be attributed to eligible product visitors. Keep assignment stable across visits, avoid overlapping page experiments, and preserve the same acquisition mix across arms. Run to the planned sample over complete weeks; do not stop on a favorable interim p-value. Report absolute and relative effects with a 95% confidence interval, alongside revenue per eligible session. Treat device results as exploratory unless separately powered.
Stop for broken purchases or tracking failures. Otherwise, monitor errors, revenue per session, refunds and contribution without changing the planned success rule. A shipping or imagery change that lifts orders but erodes contribution should not automatically roll out. Later tests must be resized against the purchase rate and traffic observed after earlier changes.
This week's deliverable is a cleanly launched button experiment with trustworthy measurement—not a claim of improved conversion after a few days. No storefront changes or experiments have been executed as part of this report.
calculations.json: funnel, weekly comparisons, sizing and impact assumptions.experiment-sizing.csv: experiment planning figures for the operations team.calculate.py: reproducible calculation script using inputs/brief.json.evidence.md: supplied evidence and its decision limits.Merchant business data is synthetic. Website observations, when present, come from the public URLs listed in the report.
There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.
Network-collected; no confirmed author source. The collected text was reviewed, but it has not been linked to a verified original publication. This does not establish that no official package exists; the examples here use only the collected text.
Use authorized inputs and keep secrets out of prompts. Review customer-facing claims, calculations, code changes, and any proposed external action before applying them.
Turn supplied funnel data into prioritized test hypotheses and variant ideas. Each hypothesis links a proposed change to an expected outcome and a reason.
Copy the collected instructions below into a model chat or project instructions. Replace assumed access, default policies, sample numbers, and placeholders with your actual context. Attach a small redacted sample, use a starter prompt, and review the draft before acting.
Find a funnel bottleneck; Write test hypotheses; Prioritize a test backlog
There is no experiment runner or sample-size implementation. Impact ranges, duration, and significance require separate calculations and observed data.
Network-collected; no confirmed author source. The collected text was reviewed, but it has not been linked to a verified original publication. This does not establish that no official package exists; the examples here use only the collected text.
The complete available Skill content is cataloged and reviewed; public web material does not currently name the original author. Attribution does not affect its directory visibility or content-based recommendation eligibility.
Network-collected; no author source is listed. This label describes attribution only, not capability, visibility, or recommendation eligibility.
No reviews yet.