How Piano Assigns a Visitor to a Variant
When a visitor first reaches an active A/B test card:
-
Piano assigns the visitor to a variant using a random weighted distribution based on your configured proportions.
-
The assignment is stored in a browser cookie (tied to that browser/device).
-
On subsequent visits, Piano reads that cookie and serves the same variant.
This behavior applies to both anonymous and authenticated users. Authentication does not override variant stickiness if the assignment is cookie-based. There is no IP-based grouping and no real-time "normalization" beyond the initial weighted assignment and the stored cookie.
What Preserves (and Breaks) Stickiness
Stickiness Is Preserved When
A visitor's variant assignment remains stable when the visitor:
-
Uses the same browser on the same device
-
Has not cleared cookies/site data
-
Is not in a mode or environment that blocks or isolates cookies
If a modal/popup is configured to reappear after a delay (for example, "don't show again for 3 hours"), and it reappears later, the visitor should still see the same variant as long as the cookie is intact.
Stickiness Breaks When
A visitor may be reassigned to a different variant when the visitor:
-
Clears cookies (or site data/cache in a way that removes the assignment cookie)
-
Switches to a different browser or device
-
Uses incognito/private browsing (often creates a new cookie jar/session)
-
Is in an environment where relevant cookies are blocked (for example, third-party cookie restrictions depending on your implementation)
When stickiness breaks, Piano treats the visitor as new for the purpose of that test and may assign a different variant.
Scope Limitation: Stickiness Is Per A/B Test
Stickiness applies to the specific A/B test a visitor encountered (and the cookie created for it). If you have:
-
Multiple A/B tests in different branches of an experience, or
-
Similar A/B tests spread across multiple experiences,
a visitor can appear to "switch variants" when they are actually encountering different tests.
Ensuring Consistency Across Pages and Experiences
If you need a visitor to remain in the same group across multiple pages or experiences (for example, to avoid showing conflicting offers/prices):
-
Piano does not automatically make A/B test assignments sticky across multiple experiences by default.
-
A common approach is to set your own variant cookie (or other persistent state) on first assignment, then:
-
Detect that cookie in subsequent experiences
-
Route the visitor to the matching offer/template/segment
-
To reduce unintended mismatch across pages, prefer using one consolidated experience across the relevant pages when possible, and/or explicitly coordinate assignment using cookies.
What Happens If You End and Re-Run a Test
If a test is completed and later re-run, it is treated as a new test instance. Prior segmentations from the old run are not retained as "the same test," and visitors may be assigned again based on the current setup and their current cookie state.
For internal QA where you need fresh assignments, use:
-
Clearing cookies/site data
-
Incognito/private browsing
-
A different browser/profile/device
Reporting: How Exposures Are Counted
Composer A/B test reporting counts exposures based on visible content.
-
Cards that display content (for example, recommendation/content cards) can generate exposure counts.
-
RunJS cards execute JavaScript but do not display content, so they typically will not show exposure counts in A/B reporting.
If you expect exposure volume but see none (or unexpectedly low counts), confirm that each A/B branch ends in a visible action.
Troubleshooting Checklist
"Only One Variant Seems to Be Live"
Most commonly, you are repeatedly seeing the same variant due to stickiness.
Try:
-
Open the site in incognito/private mode.
-
Clear cookies/site data for the site.
-
Use a different browser/profile/device.
"Visitors Are Falling Into a Fallback Segment"
If you use custom cookies/segments for cross-experience stickiness, verify:
-
The cookie is actually being set on first assignment.
-
The cookie name and expected values match exactly between:
-
The experience that sets the cookie, and
-
The segmentation rules that read it
-
-
Your segmentation logic doesn't route users to a fallback when:
-
The cookie is missing, or
-
The cookie value is unexpected
-
A high fallback percentage often indicates cookie recognition or mismatch issues.
"The A/B Test Card Isn't Being Reached"
Check the following:
-
Page targeting and segment conditions before the A/B test card
-
Exclusion logic that may prevent entry into the test
-
In the browser's Developer Tools, confirm whether the relevant cookies are present after the test should have run
"It Looks Like Users Switch Variants Within the Same Session"
This can happen when multiple A/B tests exist in the same overall flow (different branches/tests), especially if variants are named similarly.
Mitigation: Use distinct, descriptive variant names (include branch names/identifiers in your analytics tagging) to clearly differentiate which test/branch produced the variant label.
Best Practices Summary
-
Expect variant assignment to be cookie-sticky per test.
-
Use incognito/cleared cookies for QA when you need new assignments.
-
Ensure each A/B branch includes a visible action if you rely on exposure reporting.
-
If you need stickiness across experiences, implement explicit cookie-based routing (or another persistent mechanism) and keep cookie names/values consistent.
-
Use unique variant naming when multiple tests/branches exist to avoid analysis confusion.