An A/B test splits your live traffic between two or more variations so you can compare performance and pick a winner from real data - no guessing. In PDQ, A/B testing spans several surfaces, each with its own setup but the same underlying engine.
The test types
Test type | What it compares | Typical use |
Shipping Rates | Shipping Rate Groups (checkout groups) | Free shipping vs flat rate; threshold or delivery-promise strategies |
Extensions | Checkout extension blocks | Upsell vs upsell, or extension vs nothing |
Extension Groups | Whole stacks of extensions | A full checkout stack vs another (or vs nothing) |
Product Page (PDP) | Delivery-promise blocks on product pages | PDP promise placement/variants |
Cross-Site (Theme) | Entire Shopify themes | A redesigned theme, or FST + progress-bar combos |
Checkout Profile | A whole live checkout profile | 1-page vs 3-page checkout (run by our team) |
How it works
Goal metric - every test is judged on one: Checkout Conversion, AOV, Shipping Revenue, Gross Profit, or Net Revenue. (ARPC is PDQ's recommended primary metric - see PDQ Analytics Terms 101.)
Targeting - by default a test runs for everyone; add rule groups (cart value, country, customer tags, product type, and more) to limit it to a segment. When a test wins, its rules can be frozen into a segment.
Traffic split - default is 50/50; add more variants for a multi-variant test (e.g. 33/33/34). More variants need more checkout volume to reach significance.
Session stickiness (important for QA) - once a shopper sees a variant, they keep seeing it until their checkout token resets. To force a fresh allocation while testing: complete an order, or clear the cart cookie (Inspect → Application → Cookies → delete → refresh).
Managing any test
Draft → Running → Concluded tabs in Optimize → A/B Tests.
Launch: ⋯ on the draft → Start Test (goes live immediately).
Monitor: results appear in Optimize → Analytics Dashboard the day after launch.
End & pick a winner: ⋯ in Running → End Test, then implement the winner (see Implementing your winning variant).
Good to know
Some test types are gated or run by our team (Cross-Site needs a feature flag; Checkout Profile tests are run by PDQ's automation team). Your CSM can enable or launch these.