Within the LtS segment summary report, you can see a list of the top 10 LtS features (a feature is a computed metric used by a machine learning model). The features are ordered from 1-10, where 1 is the most important feature in the model in the most recent training run. This list is specific to your audience since each LtS model is trained specifically on your audience and conversion data.
LtS Model Feature Set
Below all the features used in Piano's LtS model are listed. Note that, while all features are evaluated during every training run, the model might only use 30 or 40 of the available features at any given time because highly correlated features are removed from the model to improve overall performance.
|
Feature Name |
Description |
|
Active quarter-days |
Over the last 15 days, the number of nights (00:00-05:59), mornings (06:00-11:59), afternoons (12:00-17:59), and evenings (18:00 and 23:59) with at least one pageview. |
|
Active half-days |
Over the last 15 days, the number of mornings (00:00-11:59) and evenings (12:00 and 23:59) with at least one pageview. |
|
Share Android pageviews |
Share of pageviews over the last 15 days where the Android operating system was used. |
|
Recent article pageview share (last 3 out of last 15 days) |
Share of article page views in the last 3 days vs. the last 15 days. |
|
Share pageviews on articles (last 15 days) |
Share of pageviews on articles (as opposed to non-articles URLs such as homepages and section fronts) over the last 15 days. |
|
Article pageviews per visit (last 15 days) |
Number of pageviews on articles per visit (as opposed to non-articles URLs such as homepages and section fronts) over the last 15 days. |
|
Average visit time |
Average hour of day (on 24-hour scale) when a visitor generated pageviews. |
|
Average age of content read |
Average number of days between article publish date and the date the user viewed the article. |
|
Share attractions articles |
Over the last 30 days, share of articles read classified as attractions. |
|
Share automotive articles |
Over the last 30 days, share of articles read classified as automotive. |
|
Share books and literature articles |
Over the last 30 days, share of articles read classified as books and literature. |
|
Share business and finance articles |
Over the last 30 days, share of articles read classified as business and finance. |
|
Share careers articles |
Over the last 30 days, share of articles read classified as careers. |
|
Share crime articles |
Over the last 30 days, share of articles read classified as crime. |
|
Share disasters articles |
Over the last 30 days, share of articles read classified as disasters. |
|
Share education articles |
Over the last 30 days, share of articles read classified as education. |
|
Share entertainment articles |
Over the last 30 days, share of articles read classified as entertainment. |
|
Share events articles |
Over the last 30 days, share of articles read classified as events. |
|
Share family and relationships articles |
Over the last 30 days, share of articles read classified as family and relationships. |
|
Share fine art articles |
Over the last 30 days, share of articles read classified as fine art. |
|
Share food and drink articles |
Over the last 30 days, share of articles read classified as food and drink. |
|
Share healthy living articles |
Over the last 30 days, share of articles read classified as healthy living. |
|
Share holidays articles |
Over the last 30 days, share of articles read classified as holidays. |
|
Share home and garden articles |
Over the last 30 days, share of articles read classified as home and garden. |
|
Share law articles |
Over the last 30 days, share of articles read classified as law. |
|
Share medical health articles |
Over the last 30 days, share of articles read classified as medical health. |
|
Share personal celebrations articles |
Over the last 30 days, share of articles read classified as personal celebrations and life events. |
|
Share personal finance articles |
Over the last 30 days, share of articles read classified as personal finance. |
|
Share politics articles |
Over the last 30 days, share of articles read classified as politics. |
|
Share pop culture articles |
Over the last 30 days, share of articles read classified as pop culture. |
|
Share real estate articles |
Over the last 30 days, share of articles read classified as real estate. |
|
Share science articles |
Over the last 30 days, share of articles read classified as science. |
|
Share sports articles |
Over the last 30 days, share of articles read classified as sports. |
|
Share style and fashion articles |
Over the last 30 days, share of articles read classified as style and fashion. |
|
Share technology and computing articles |
Over the last 30 days, share of articles read classified as technology and computing. |
|
Share television articles |
Over the last 30 days, share of articles read classified as television. |
|
Share travel articles |
Over the last 30 days, share of articles read classified as travel. |
|
Share video gaming articles |
Over the last 30 days, share of articles read classified as video gaming. |
|
Share war and conflict articles |
Over the last 30 days, share of articles read classified as war and conflict. |
|
Share Chrome pageviews |
Share of page views in the last 15 days where Chrome browser was used. |
|
Maximum time between paid offers |
Number of days between the first and last paywall hit over the last 30 days. |
|
Share desktop pageviews |
Share of pageviews over the last 15 days where desktop device was used. |
|
Share direct desktop pageviews (last 15 days) |
Share of pageviews over the last 15 days where the referrer was direct and the device was desktop. |
|
Direct visits share |
Over the past 15 days, share of visits with direct referrer. |
|
Share evening pageviews |
Share of pageviews over the last 15 days during evening time (relative to user's timezone). |
|
External visits share |
Share of pageviews over the last 15 days referred from any non-direct source (including search, social, and other referral categories). |
|
Share Firefox pageviews |
Share of pageviews over the last 15 days where Firefox browser was used. |
|
Share recently published articles read |
Out of all articles read, share of articles read the same day they were published. |
|
Paywall hit frequency |
Days between subscription offers divided by the number of active days with subscription offers over the last 30 days. |
|
Breadth of visit times |
Range of hours during which page views were made over the last 15 days. |
|
Share pageviews from recirculation (last 15 days) |
Share of pageviews over the last 15 days from recirculation. |
|
Internal visits share |
Share of visits over the last 15 days with internal referrer. |
|
iOS pageviews share |
Share of pageviews over the last 15 days where iOS operating system was used. |
|
Linux pageviews share |
Share of pageviews over the last 15 days where Linux operating system was used. |
|
Latest hour visited |
Latest hour at which pageviews were made over the last 15 days. |
|
Median age of content read |
Median number of days between article publish date and visitor pageview. |
|
Earliest hour visited |
Earliest hour at which pageviews were made over the last 15 days. |
|
Mobile pageviews share |
Share of pageviews over the last 15 days where mobile device was used. |
|
Morning pageviews share |
Share of pageviews over the last 15 days during morning time (relative to user's time zone). |
|
Share Internet Explorer pageviews |
Share of pageviews over the last 15 days where Internet Explorer operating system was used. |
|
Number of devices |
Number of unique devices used over the last 30 days. |
|
Unique subdomains visited |
Number of unique subdomains of a website visited over the last 30 days. |
|
Unique referrers |
Number of unique referer classes (Direct, Search, Social, External, Internal) a visitor has used over the last 30 days. |
|
Opera pageviews share |
Share of pageviews over the last 15 days where Opera operating system was used. |
|
Share other referrers |
Share of pageviews over the last 15 days from "other" referrer (not search, social, direct, or internal). |
|
Other referrer visits share |
Share of visits over the last 15 days from "other" referrer (not search, social, direct, or internal). |
|
Share of active days with paid offer |
Out of a visitor's total active days over the past 30 days, the ratio of days where a paid offer was seen. |
|
Paywall hit intensity |
Number of paywall hits per day with paywall hits. |
|
Pageview intensity during active period (last 15 days) |
Total number of pageviews divided by the number of days between the first and the last pageview over the last 15 days. |
|
Pageview intensity during active period (last 3 days) |
Total number of pageviews divided by the number of days between the first and the last pageview over the last 6 days. |
|
Pageview intensity during active period (last 30 days) |
Total number of pageviews divided by the number of days between the first and the last pageview over the last 30 days. |
|
Recent pageview share (last 15 out of last 30 days) |
Share of pageviews in the last 15 days vs. the last 30 days. |
|
Share of pageviews in the last 3 days |
Out of all pageviews over the last 15 days, the share from the last 3 days. |
|
Pageviews per active day (last 15 days) |
Number of pageviews per active day in the last 15 days. |
|
Pageviews per active day (last 3 days) |
Number of pageviews per active day in the last 3 days. |
|
Pageviews per active day (last 30 days) |
Number of pageviews per active day in the last 30 days. |
|
Pageviews per visit (last 15 days) |
Number of pageviews per visit in the last 15 days. |
|
Time between first and last pageview (last 15 days) |
Number of days between the first observed pageview and the last observed pageview over the last 15 days. |
|
Time between first and last pageview (last 3 days) |
Number of days between the first observed pageview and the last observed pageview over the last 3 days. |
|
Time between first and last pageview (last 30 days) |
Number of days between the first observed pageview and the last observed pageview over the last 30 days. |
|
Safari pageviews share |
Share of pageviews over the last 15 days where the Safari operating system was used. |
|
Click velocity |
Variation in time between pageviews over the last 15 days. |
|
Search visits share |
Share of visits over the last 15 days referred from search. |
|
Social visits share |
Share of visits over the last 15 days referred from social media. |
|
Share tablet pageviews |
Share of pageviews over the last 15 days where tablet device was used. |
|
Author regularity (last 15 days) |
Unique authors divided by the number of pageviews over the last 15 days. |
|
Author regularity (last 3 days) |
Unique authors divided by the number of pageviews over the last 3 days. |
|
Author regularity (last 30 days) |
Unique authors divided by the number of pageviews over the last 30 days. |
|
Active days (last 15 days) |
Number of active days in the last 15 days. |
|
Active days (last 3 days) |
Number of active days in the last 3 days. |
|
Active days (last 30 days) |
Number of active days in the last 30 days. |
|
Active days with paid offer |
Number of unique days the visitor viewed a paid offer. |
|
Visit frequency |
Days between visits divided by the number of active days over the last 15 days. |
|
Ratio recent visits (last 3 days out of 15 days) |
Share of visits in the last 3 days vs. the last 15 days. |
|
Visits per day (last 15 days) |
Number of visits per day in the last 15 days. |
|
Share non-article pageviews |
Share of pageviews over the last 15 days on general website URLs (such as homepage or section fronts). |
|
Share weekday pageviews |
Share of pageviews on weekdays in the last 15 days. |
|
Share weekend pageviews |
Share of pageviews on weekends in the last 15 days. |
|
Share Windows pageviews |
Share of pageviews in the last 15 days with Windows operating system. |
Data Science Download
For the data scientists on your team, the LtS segment summary report includes a detailed export that can be downloaded by clicking the down arrow in the upper right-hand corner of the report and then selecting "download model data." This downloads a JSON with key details on model training and performance that should be valuable to any data scientists on your team.
JSON Example
Here is an example of what the JSON will look like:
{
"best_iteration": 529,
"correlations": { ... },
"last_updated": "2022-08-04T17:38:49+02:00",
"parameters": {
"optimization_method": "bayesianOptimization",
"parameters": { ... }
},
"train_traffic_period": {
"start": 1656892803,
"stop": 1659570572
},
"total_running_time": 5881.687784433365,
"feature_importance": { ... },
"distribution": {
"0": 0.09744162511175047,
"1": 0.0834267011981444,
...
"no_score": 0.6190561589125827
},
"precision": 0.7831182795698924,
"recall": 0.7283
}
Data Science Definitions
To help your data science team make sense of the JSON, here are descriptions related to elements of the JSON.
-
best_iteration The best iteration obtained by early stopping. In order to avoid overfitting, early stopping is activated during training. The 'best_iteration' parameter represents the number of iterations conducted during model training. Read more here.
-
correlations Pearson's correlation coefficients between features and the target variable. Pearson's correlation coefficient measures linear correlation. It is a number between -1 and 1 that measures the strength and direction of the relationship between two variables. Values between |0.7| and |1| indicate strong correlation, values between |0.4| and |0.7| medium correlation, and values below |0.4| indicate weak or no correlation. Read more here.
-
last_updated The timestamp of the last successful model training run. Model training runs are scheduled to run every 24 hours. However, Piano has automated quality checks that prevent bad quality models going into production, in which case the last successfully trained model is used.
-
parameters/optimization_method Hyper-parameter optimization method. Currently, the only optimization method implemented is bayesian hyper-parameter optimization. Read more here.
-
parameters/parameters XGBoost parameters. Optimal parameters for the XGBoost algorithm. Results of hyper-parameter optimization.
-
parameters/parameters/colsample_bytree The subsample ratio of columns when constructing each tree. The optimal ratio of columns used for constructing each tree during XGBoost training. The parameter's optimal value is searched in the range of (0.5, 1). Read more here.
-
parameters/parameters/eta Learning rate parameter. Step size shrinkage used to prevent overfitting. After each boosting step, eta shrinks the feature weights to make the boosting process more conservative. The parameter's optimal value is searched in the range of (0.001, 0.1). Read more here.
-
parameters/parameters/gamma Minimum split loss parameter. Minimum loss reduction required to make a further partition on a leaf node of the tree. The parameter's optimal value is searched in the range of (0, 5). Read more here.
-
parameters/parameters/max_depth Maximum depth of a tree. Increasing the value of this parameter makes the model more complex and more likely to overfit, lower values indicate a more robust approach. The parameter's optimal value is searched in the range of (2, 4). Read more here.
-
parameters/parameters/min_child_weight Minimum sum of instance weight in a child. The parameter's optimal value is searched in the range of (5, 10). Read more here.
-
parameters/parameters/n_estimators Number of boosting rounds. The parameter's optimal value is searched in the range of (100, 1000). Read more here.
-
parameters/parameters/subsample Subsample ratio of the training instances. The parameter's optimal value is searched in the range of (0.5, 1). Read more here.
-
parameters/parameters/reg_alpha L1 regularization term on weights. Increasing this value will make model more conservative. The value of this parameter is set to 0.01. Read more here.
-
train_traffic_period The time period for training data selection. Start represents the oldest pageview event included, stop represents the most recent pageview event included in building the training dataset.
-
total_running_time Run time of the training job. The run time of the most recent training job in seconds.
-
feature_importance List of features and their feature importance score. Importance type used is 'gain': the average gain across all splits the feature is used in.
-
distribution Estimated share of visitors in each segment. The estimates are calculated on the validation dataset.
-
precision Model precision. The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. Read more here.
-
recall Model recall. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. Read more here.
LtS Algorithm & Report General Overview
For information on requirements to activate the LtS model, details on how to use the LtS model in the Composer interface, and FAQs on the nature of the algorithm itself, please reference this documentation. For more information on the LtS segment summary report, please reference this documentation.