An A/B test compares two experiences side by side to see which performs better. Instead of guessing, you split real customer traffic between versions and let the data decide. PDQ runs the test automatically - set it up, launch it, check the results.
An Extensions A/B Test compares two checkout extensions against each other - for example, a Checkout Upsell with free-shipping messaging vs one with a discount badge. One extension is the control, the other is the test variant.
You can also test nothing vs. an extension - one group sees no extension, the other sees your chosen extension - to measure its true incremental impact.
Note: Any display rules already configured on each extension still apply during the test - the test respects those settings, it doesn't override them.
Step 1: Create the test
Go to Optimize → A/B Tests. Click New Test and select Extensions Test.
Step 2: Fill in the details
Test Name - e.g. "Upsell with FST vs Upsell with Discount".
Hypothesis - min. 100 characters: "Showing a free-shipping-threshold message will increase AOV more than a discount badge by motivating customers to add one more item."
End Test After X Checkouts - optional.
Goal Metric - Checkout Conversion, Gross Profit, AOV, Shipping Revenue, or Net Revenue.
Click Next.
Step 3: Set rules (optional)
Add conditions to limit who enters the test (cart value, product type, location, etc.). Leave empty to run for everyone. Click Next.
Step 4: Allocate traffic
Extension vs. Extension - pick a Control and a Test extension; set the split (typically 50/50). Test apples-to-apples (upsell vs upsell), never mismatched block types.
Nothing vs. Extension - set Control to NULL (no extension), Test to your chosen extension.
Click Create draft test.
Step 5: Connect to Shopify and launch
A Token for Shopify tab appears in the test. Which token(s) to enter depends on the setup:
Scenario | What to enter in Shopify |
One block (upsell vs upsell in one spot, or block vs none) | Test ID only |
Two blocks / two placements / different types | Usage token (per block) + Test ID |
Block vs none with a fallback | Test ID + usage token for the "show" variant, plus enable the default block option with a fallback usage token (don't leave it empty) |
Add a token in Shopify via Settings → Checkout → Customize → select your checkout → Add block → paste the token. PDQ then serves the correct variant per shopper.
⚠️ Don't double up tokens. Never put a token in both "Enter desired token" and the "AB Test Configuration" field on the same extension - it fires two webhook responses and makes the extension display inconsistently (a known Gift Unlock issue). Use the AB Test Configuration field only.
Then find your test in Drafts → ⋯ → Start Test.
Managing your test
Editing a draft - Drafts → ⋯ → Edit.
Launching - ⋯ → Start Test.
Monitoring - Running tab; data in Analytics Dashboard the day after launch.
Ending - Running → ⋯ → End Test; results in Analytics Dashboard, or Concluded → ⋯ → Get Analytics. To roll out the winner, see Implementing your winning variant.
QA tip: assignment is sticky per checkout token - complete an order or clear the cart cookie for a fresh allocation.
Happy testing! Reach out to support@prettydamnquick.com if you need help.