Return Model Details
Piano's Likelihood to Return model uses machine learning to detect patterns of behavior among loyal and non-loyal visitors in order to create propensity scores. These scores indicate how likely a visitor is to be amongst the most loyal visitors to the site within the next seven days on a scale from 0 to 100, with a higher score indicating a higher likelihood to be loyal. As a consequence, the highest scored segments have the highest number of visits and pageviews.
Much like Piano's other propensity solutions, this algorithm uses behavioral and semantic data. The data and features in the model overlap significantly with Piano's Likelihood to Subscribe (LtS) model, and include behavioral characteristics such as:
-
Contextual data like referrer and device
-
Behavioral data like article consumption, time of day, active days
-
Preference data like content categories of articles consumed
The total number of features considered in the model is around 100. During the learning phase, the most informative subset of features for your website is used for training your model. In order to make the model more robust, a custom set of hyper-parameters, selected based on your training data, is used for optimizing the model's performance. Every time a visitor on the site is targeted by a campaign that involves the propensity score, their propensity score is computed in real-time, based on the latest trained model. The model itself is re-trained daily.
An important learning, from clients running return propensity alongside subscription propensity, is there are often high return likelihood visitors who are low subscription likelihood. While there is a generally positive relationship between loyalty and subscription conversion, this relationship is not one-to-one and there are frequently high loyalty visitors who are not high subscription likelihood.
One reason for this is the device's role in loyalty and conversion. Subscription propensity tends to pick up on the fact that desktop visitors have higher conversion likelihood, which makes sense given that desktop visitors typically have much higher conversion rates than mobile visitors. This results in a greater share of high LtS visitors coming from desktop.
But loyalty often does not have this kind of device skew, with mobile visitors often as loyal, or more loyal, than desktop visitors.
Model Requirements
Return propensity is enabled by default for Piano customers if the below requirements are met.
Before real-time return propensity is available, there are a few prerequisites:
-
Composer 1X implementation
-
At least 31 days of page view events sent to Piano post-implementation
Unlike other Piano propensity solutions, such as subscription propensity or registration propensity, there is no need for you to send conversion data to train the model since visitation data is automatically collected once the Composer 1X implementation is complete.
Audience Scoring Triggers
The model calculates in real time the propensity score for a visitor as of the second pageview in the last 31 days — meaning that segmentation based upon propensity score can begin as early as the visitors' second pageview in the past 31 days. Propensity is then re-calculated on each page view. One-off visitors (often ~70% of the audience) and those visitors who did not provide GDPR consent are not scored. For visitors without propensity scores, experiences can be configured using Composer's standard drag-and-drop logic.
Propensity Score Ranges
After Piano's return propensity model creates scores for visitors on a 0 to 100 scale, those scores are then broken into 10 segments that can be targeted using Composer 1X. The names of those ten segments (11 segments including "no score"):
-
0-9
-
10-19
-
20-29
-
30-39
-
40-49
-
50-59
-
60-69
-
70-79
-
80-89
-
90-100
-
No score
Visitors in the 0-9 range have the lowest likelihood to be loyal. Visitors in the 90-100 range have the highest likelihood to be loyal. The distribution of visitors between the groups isn't equal — there will typically be a few percent of visitors in the highest propensity segment that contribute an outsized share of pageviews. Piano's model is trained custom based on an individual site's data, which means the exact distribution varies from site to site.
The "no score" segment shows users which we do not expect to score. If a visitor retains a "no score" beyond pageview 2, it is likely that visitor is using an ad blocking solution that is interfering with data capture. You may want to consider targeting repeat "no score" visitors differently from one or two-time "no score" visitors.
Composer Setup
Once return propensity is enabled on your site, and there are sufficient days of training data, you will be able to select segments using the Composer 1X segmentation engine in the user segmentation card:
You can then select which segments you want to target individually, or as groups of segments. You can also target visitors who don't have propensity scores by selecting all subscription propensity segments and moving the toggle to "ignore" rather than "target". Or alternatively by selecting only the "no score" segment and choosing the option "target".
When setting up experiences, Composer 1X reporting will show you an estimation of the number of visitors expected on each segment to help you decide which segments are best to select for each campaign.
Return Propensity Data Capture
Should you desire to ingest return propensity data into an external system, you can follow the same process that is described here for capturing LtS data.
In-product reporting available for subscription propensity, such as the LtS Segment Summary, is not currently available for return propensity. However, with a paid services engagement, Piano's data analyst and data science team can manually extract data of a similar nature to the LtS Segment Summary report for return propensity.