A/B 测试假设生成器

HarborMug: funnel diagnosis and experiment plan

Start with the purchase button this week. Put the existing button immediately below price and above care information, and test that placement against the current page. Order growth alone does not show that the product page improved: orders and traffic both rose 20%, while conversion stayed at 2.50%.

The supplied analytics and interface descriptions are synthetic; all experiment lifts and revenue effects below are planning assumptions, not measured results. No live product page or historical product-page funnel was supplied.

What the numbers say

Metric	Prior week	Current week	Change
Sessions	4,000	4,800	+20%
Orders	100	120	+20%
Session-to-order conversion	2.50%	2.50%	0 percentage points
Gross product revenue	$2,400	$2,880	+20%
Average order value	$24	$24	Unchanged
Gross revenue per session	$0.60	$0.60	Unchanged
Refund amount / gross revenue	5%	5%	Unchanged
Ad spend per order	$3	$3	Unchanged
Modeled contribution after ads*	$780	$936	+20%

Gross product revenue minus refunds, $8 unit cost per order, $4 merchant shipping cost per order and supplied ad spend. This assumes one unit per order and excludes collected shipping fees, taxes, payment fees, overhead and any returned-stock recovery. It is not accounting profit. Refund amounts may relate to earlier orders; the 5% ratio is not a cohort return rate.

At the prior 2.50% conversion rate, 4,800 sessions produce exactly 120 orders. Increased traffic fully explains the observed order increase arithmetically. This does not establish what caused traffic growth or rule out offsetting changes within devices. A prior product funnel and randomized comparison are missing, so the team cannot establish a product-page improvement from these totals.

Current funnel

Step	Entered	Progressed	Step conversion	Lost	Step drop-off
Session → product	4,800	3,600	75.00%	1,200	25.00%
Product → cart	3,600	720	20.00%	2,880	80.00%
Cart → checkout	720	360	50.00%	360	50.00%
Checkout → order	360	120	33.33%	240	66.67%

Product-session-to-order conversion is 120 / 3,600 = 3.33%. Product → cart has both the largest absolute loss and the highest proportional loss. Checkout also deserves attention: two-thirds of checkout sessions do not become orders, and the supplied interface description reports a long single-column form. That is a separate investigation, not a change to bundle into these tests.

Page	Views	Exits	Exit rate
Product	4,200	2,100	50%
Cart	900	270	30%
Checkout	450	180	40%

Product has the highest reported exit rate. Page views include repeat views; these exit rates are not funnel drop-off rates and should not be substituted for them.

Device	Sessions	Orders	Conversion
Mobile	3,600	60	1.67%
Desktop	1,200	60	5.00%

Mobile supplies 75% of sessions but only 50% of orders; its conversion is one-third of desktop's. This makes mobile usability the first diagnostic priority, but traffic quality or customer mix could also explain the gap. Device-level product sessions and cart progression were not supplied.

Three prioritized experiments

The primary success metric for all three is completed purchases / eligible product-page sessions, baseline 3.33%. Use add-to-cart and checkout progression as diagnostic metrics, not substitutes for purchase success. The common denominator allows revenue and test duration to be compared consistently.

Rank	Experiment	Assumed purchase lift	Full-rollout added orders/week	Full-rollout added gross revenue/week	Added contribution/week*	Ease (1–5)	Priority score
1	Purchase-button placement	15–25% relative	18–30	$432–$720	$194–$324	5	2,880
2	Shipping information	10–20% relative	12–24	$288–$576	$130–$259	4	1,728
3	Size imagery	5–15% relative	6–18	$144–$432	$65–$194	2	576

Priority score = midpoint weekly gross revenue impact × implementation ease. Ease is a planning judgment, with 5 easiest. These are positive planning scenarios; actual impact could be zero or negative. Confidence in every numeric lift is Low, since the brief contains no experiment or customer research evidence. The button has the strongest directional rationale because the reported mobile placement and funnel loss align.

Incremental contribution assumes $24 × 95% − $8 − $4 = $10.80 per additional order, with ad spend held constant and the same exclusions as above. Revenue scenarios assume the existing 120 weekly orders all pass through the reported product funnel and that traffic, AOV and refund ratio stay constant. During a 50/50 test, the expected incremental store impact is about half the full-rollout estimate. The experiments overlap and their impacts must not be added together.

1. Purchase-button placement — quick win, under one day

Customer problem: On mobile, the buy button sits below care copy. A shopper ready to buy may not immediately see how to proceed.

Hypothesis: If the existing purchase button appears immediately below price and above care copy, then more product sessions will become purchases, because ready-to-buy shoppers can find the next action sooner.

• Control: Existing page; mobile buy button below care copy. Capture the actual desktop placement before launch because it is unspecified.
• Variant A — smallest useful change: Move the existing button above care copy, immediately below price. Keep its existing label, styling, price and behavior. Keep care details immediately beneath it, including “Hand-wash only. Not microwave safe.” Apply the placement rule across eligible devices; record whether desktop changes at all.
• Variant B — bolder alternative: A persistent mobile purchase bar with price and the existing purchase action, using the same cart behavior. Proposed copy: “Add to cart — $24”, only if the action actually adds to cart. This is a future alternative, not an additional arm this week.
• Baseline and target: Product-to-cart 20% for diagnosis; purchase rate 3.33%, targeting 3.83–4.17% in the planning scenario. Monitor purchase errors and accidental/duplicate cart additions.
• Sample and duration: At a 25% relative purchase lift, 8,158 sessions per arm; 16,316 total; about 4.53 weeks, budget 5 full weeks. Expected full-rollout revenue impact: +$432–$720/week. Lift confidence: Low.

The all-device lift assumption is ambitious if only mobile presentation changes. Report mobile results separately, but do not claim a mobile-specific winner from this pooled sample plan. Mobile-only sizing requires mobile product-session volume and its purchase rate.

2. Shipping information — quick win, under one day

Customer problem: The cart first reveals shipping charges. Shoppers cannot assess cost and timing before committing to the next step.

Hypothesis: If the product page states shipping price, eligibility and timing beside the purchase action, then more product sessions will become purchases, because shoppers can judge the delivered offer before entering the cart.

• Control: Shipping charge first revealed in cart.

• Variant A — smallest useful change: Add a compact text block directly below the purchase button without changing rates or delivery promises:

  Domestic shipping: $5 on orders under $50; free on orders of $50 or more. Dispatch within 2 business days. Delivery takes 3–5 business days after dispatch. No international delivery.

  Confirm that the stated delivery window begins after dispatch before publishing; the supplied policy lists dispatch and delivery separately but does not explicitly define that timing relationship. If unconfirmed, use the literal policy wording: “Domestic dispatch: 2 business days. Delivery: 3–5 business days.” Keep the remaining shipping text above unchanged.

• Variant B — bolder alternative: An expandable “Shipping & delivery” panel beside the action, displaying the current cart's actual shipping charge and the amount remaining to reach $50. Example for a one-mug cart: “Shipping: $5. Add $26 to reach free shipping.” Recalculate from actual cart value; do not change the threshold.

• Baseline and target: Purchase rate 3.33%, targeting 3.67–4.00%. Cart-to-checkout is 50%, a secondary diagnostic. Earlier disclosure may reduce cart additions while improving purchase quality, so judge purchases first. Monitor gross revenue and contribution per exposed session.

• Sample and duration: At a 20% relative lift, 12,475 sessions per arm; 24,950 total; about 6.93 weeks, budget 7 full weeks. Expected full-rollout revenue impact: +$288–$576/week. Lift confidence: Low.

3. Size imagery — medium effort, 1–3 days

Customer problem: One studio image gives little context for the mug's physical scale, potentially leaving shoppers unsure whether it fits their intended use.

Hypothesis: If the gallery shows verified dimensions and an accurate scale reference, then more product sessions will become purchases, because shoppers can better judge the mug's size.

• Control: Single studio image.
• Variant A — smallest useful change: Add one dimension image as the second gallery item. Draft caption: “Dimensions: 9 × 9 × 19 cm.” Place arrows only after confirming which measurement belongs to each axis and whether dimensions include the handle or other protrusions. No capacity or compatibility claim is supported by the brief.
• Variant B — bolder alternative: A two-image scale sequence: the dimension image plus a real, accurately scaled in-hand photo. Draft caption: “See the size in hand.” Production estimate: 1–2 weeks if photography must be commissioned. Do not use an unverified illustration as evidence of scale.
• Baseline and target: Purchase rate 3.33%, targeting 3.50–3.83%. Product-to-cart baseline is 20%; monitor gallery engagement diagnostically and later size-related returns as a guardrail. No historical size-return baseline is available.
• Sample and duration: At a 15% relative lift, 21,693 sessions per arm; 43,386 total; about 12.05 weeks, budget 13 full weeks. Expected full-rollout revenue impact: +$144–$432/week. Lift confidence: Low.

The physical dimension mapping and scale assets are not supplied. These are completed variant specifications and copy drafts; an accurate customer-facing image cannot be finished from the brief alone.

How to run and read the tests

Sample sizes use a two-sided two-proportion normal approximation, 5% significance, 80% power and 50/50 allocation. Each calculation compares control against Variant A only, using the upper end of that experiment's assumed lift range as the minimum effect the test is powered to detect. Smaller lifts require more traffic; a nonsignificant result at these sample sizes would not rule out the lower end of the range. Variant B is an alternative concept, not covered by these sample sizes or impact estimates; size it separately before use.

Duration assumes 3,600 eligible product sessions per week, or 1,800 per arm, no traffic ramp and no exclusions. These are three sequential planning estimates, not a promise that all three can finish in parallel. Repeated sessions from the same shopper are not independent: assign visitors persistently, analyze at the randomization unit and recalculate sample needs from deduplicated eligible visitors before launch. Session-based figures here are provisional because unique-visitor counts are absent.

Use a consistent purchase-attribution window, fixed before launch; allow that window to mature after exposure ends. Verify that orders can be attributed to eligible product visitors. Keep assignment stable across visits, avoid overlapping page experiments, and preserve the same acquisition mix across arms. Run to the planned sample over complete weeks; do not stop on a favorable interim p-value. Report absolute and relative effects with a 95% confidence interval, alongside revenue per eligible session. Treat device results as exploratory unless separately powered.

Stop for broken purchases or tracking failures. Otherwise, monitor errors, revenue per session, refunds and contribution without changing the planned success rule. A shipping or imagery change that lifts orders but erodes contribution should not automatically roll out. Later tests must be resized against the purchase rate and traffic observed after earlier changes.

What to do this week

1. First day: Verify product-view, assignment, add-to-cart, checkout and purchase events; reconcile attributed purchases with the supplied funnel structure. Capture current mobile and desktop button placement, count eligible unique visitors and confirm the purchase-attribution window. This resolves whether the provisional sample plan is usable.
2. Build and launch button Variant A once tracking is verified. QA narrow mobile screens, desktop, cart behavior and visibility of care restrictions. Split eligible visitors 50/50 and keep all other page elements fixed. Budget approximately five weeks at the supplied session volume, subject to deduplication and attribution lag.
3. Prepare the shipping copy and dimension-image brief while the button test runs. Confirm delivery timing and dimension axes. Queue shipping next, then reassess whether a roughly 13-week imagery test is worth the opportunity cost at the updated baseline.

This week's deliverable is a cleanly launched button experiment with trustworthy measurement—not a claim of improved conversion after a few days. No storefront changes or experiments have been executed as part of this report.

Supporting files

• calculations.json: funnel, weekly comparisons, sizing and impact assumptions.
• experiment-sizing.csv: experiment planning figures for the operations team.
• calculate.py: reproducible calculation script using inputs/brief.json.
• evidence.md: supplied evidence and its decision limits.