Cross-Site Testing lets you A/B test entire Shopify themes against each other - not isolated elements, but complete customer journeys. Traffic is split at the storefront level (e.g. 50% of shoppers see Theme A, 50% see Theme B), and each theme can be paired with its own checkout experience inside PDQ. This isn't page-level testing - it's full-experience testing.
Why Cross-Site Testing exists
Most A/B tools optimize fragments - a button, a banner, a headline. But shoppers don't experience fragments; they experience a flow: land → browse → add to cart → checkout → purchase. When the storefront's promise and the checkout reality don't match, trust breaks.
Example failure: the theme promotes "Free Shipping," but checkout shows $9.99 shipping. The shopper feels misled, and conversion drops. Cross-Site Testing removes that disconnect by keeping the story consistent from first page to payment confirmation.
What makes it different
Cross-Site Testing pairs each theme with a matching checkout experience:
Theme | Storefront message | Checkout experience |
Theme A | Standard pricing | Standard shipping rates |
Theme B | Free-shipping promotion | Free-shipping checkout group |
This unlocks tests that were previously impossible:
Free-shipping themes vs standard-pricing themes
Premium brand redesigns vs legacy layouts
Aggressive promos vs margin-protected experiences
You're no longer testing components - you're testing business strategies. It's also the right fit for a free-shipping-threshold + progress-bar combo that needs full theme-level splitting rather than a single extension.
Before you start
Cross-Site Testing must be enabled for your store (the enableCrossSiteAbTest feature flag). If you don't see the Cross-Site option, ask your CSM to switch it on. Have your duplicate/variant theme ready in Shopify.
Step 1: Create the test
Go to Optimize → A/B Tests. Click New Test and select Cross-Site Test.
Step 2: Fill in the details
Test Name - e.g. "New Theme Redesign vs Current Live Theme."
Hypothesis - what you expect to learn (min. 100 characters): "The new theme redesign will improve checkout conversion by providing a cleaner, more streamlined experience."
Goal Metric - Checkout Conversion, Gross Profit, AOV, Shipping Revenue, or Net Revenue.
Click Next.
Step 3: Allocate traffic
For each theme in the test:
Set the traffic percentage (all included themes must total 100%).
Assign a Checkout Group - this determines the PDQ checkout experience served alongside that theme.
Make sure each theme's checkout group matches the experience you want that group to see, and verify the PDQ Checkout Group is linked to the live theme, not just the variant. Save as a draft - don't launch yet.
Step 4: Turn on "PDQ THEMES" for every theme
For PDQ to route traffic across themes, the "PDQ THEMES" toggle must be ON in the Shopify theme editor for every theme in the test (control + all variants). Do this only when you're ready to go live. (Behind the scenes, the test writes a cart attribute - pdq_theme_checkout_groups = A/B/C - so the correct checkout experience follows each theme.)
Step 5: Launch the test
Back in PDQ, find your test in Drafts → ⋯ → Start Test. It only routes traffic correctly when the test is Running AND the PDQ THEMES toggle is ON for all themes.
⚠️ Concluding a Cross-Site test - read this first
Select the winning theme in PDQ; traffic then auto-routes to it. Do not delete the losing/variant theme before selecting a winner - deleting a variant mid-test leaves a persistent preview_theme_id cookie in shoppers' browsers that Shopify can't clear, which can strand those shoppers on the deleted theme. Pick the winner first, then clean up.
Managing your test
Editing a draft - Drafts → ⋯ → Edit.
Launching - ⋯ → Start Test.
Monitoring - a Running tab appears once live; data lives in Optimize → Analytics Dashboard, available the day after launch.
Ending - Running → ⋯ → End Test, then select the winning theme. Results in the Analytics Dashboard, or Concluded → ⋯ → Get Analytics. To roll out the winner, see Implementing your winning variant.
QA tip: theme assignment is sticky per session - complete an order or clear the store's cart cookie and refresh to see a fresh allocation.
Happy testing! Reach out to support@prettydamnquick.com